Bug 1507030
Summary: | Subscription-manager returns "System certificates corrupted. Please reregister." when listing available subscriptions | ||||||||
---|---|---|---|---|---|---|---|---|---|
Product: | Red Hat Enterprise Linux 7 | Reporter: | Alexander Rydekull <arydekul> | ||||||
Component: | subscription-manager | Assignee: | Alex Wood <awood> | ||||||
Status: | CLOSED ERRATA | QA Contact: | Red Hat subscription-manager QE Team <rhsm-qe> | ||||||
Severity: | medium | Docs Contact: | |||||||
Priority: | medium | ||||||||
Version: | 7.4 | CC: | arydekul, awood, csnyder, hasuzuki, jhnidek, jhradile, jsefler, khowell, redakkan, wpoteat | ||||||
Target Milestone: | rc | Keywords: | Triaged | ||||||
Target Release: | --- | ||||||||
Hardware: | x86_64 | ||||||||
OS: | Linux | ||||||||
Whiteboard: | |||||||||
Fixed In Version: | Doc Type: | If docs needed, set a value | |||||||
Doc Text: | Story Points: | --- | |||||||
Clone Of: | Environment: | ||||||||
Last Closed: | 2020-09-29 19:22:40 UTC | Type: | Bug | ||||||
Regression: | --- | Mount Type: | --- | ||||||
Documentation: | --- | CRM: | |||||||
Verified Versions: | Category: | --- | |||||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
Cloudforms Team: | --- | Target Upstream Version: | |||||||
Embargoed: | |||||||||
Bug Depends On: | |||||||||
Bug Blocks: | 1796188 | ||||||||
Attachments: |
|
Description
Alexander Rydekull
2017-10-27 13:02:39 UTC
Created attachment 1344304 [details]
rhsm.log from subscription attempts.
I am experiencing the very same issue, also with the Employee SKU, except for me, the subscription-manager also randomly throws in a complaint object serialization: [jhradilek@server ~]$ sudo subscription-manager register Registering to: subscription.rhsm.redhat.com:443/subscription Username: jhradile Password: The system has been registered with ID: cc3629b3-ede3-481a-b269-ba63a501d6cb [jhradilek@server ~]$ sudo subscription-manager list --available Unable to serialize objects to JSON. [jhradilek@server ~]$ sudo subscription-manager list --available System certificates corrupted. Please reregister. That this is a fresh installation of RHEL 7.4 Server installed using the rhel-server-7.4-x86_64-dvd.iso image downloaded from the Customer Portal. Please note that the command I used is actually recommended by our official Product Documentation: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system_administrators_guide/chap-subscription_and_support-registering_a_system_and_managing_subscriptions#sect-Subscription_and_Support-Registering_a_System_and_Managing_Subscriptions-Registering_the_System_and_Attaching_Subscriptions I want to hope that our customers are not affected by this. Has anybody looked into this yet? Alexander, thank you for your workaround! It didn't occur to me to try using --matches to limit the number of entries, and it indeed solves the problem. Jaromir, Are you running subscription-manager in a container? From Alexander's comments it looks like he was running subscription-manager in a container which isn't really supported (since subscription-manager uses DBus and a Docker container won't have a DBus daemon running). If you are running outside a container, would you mind reproducing the issue and then attaching the /var/log/rhsm/rhsm.log file? I've tried to reproduce this issue with my own account that has an Employee SKU entitlement, but didn't encounter the problem. Hi Alex, Thank you for looking into this. I was not running subscription-manager in a container, but a fresh installation of RHEL 7.4 Server running in a virtual machine for testing purposes (in QEMU/KVM to be precise). I tried to reproduce the error today in the same virtual machine and was prompted to reregister the system by using the following two commands: sudo subscription-manager clean sudo subscription-manager register After doing so, I can get the complete list of available subscriptions without running into the error I reported. This didn't work in November. I am, however, still not able to attach the subscription I want, but that might be a different problem: [jhradilek@server ~]$ subscription-manager attach --pool 8a85f98260c27fc50160c323263339ff You are attempting to run "subscription-manager" which requires administrative privileges, but more information is needed in order to do so. Authenticating as "root" Password: Runtime Error could not extract ResultSet at com.mysql.jdbc.SQLError.createSQLException:1,078 I am going to attach my /var/log/rhsm/rhsm.log file which includes records from both today and November. Created attachment 1380434 [details]
The /var/log/rhsm/rhsm.log file
Jaromir, For your problem in November, I see the following in the log file: 2017-11-21 20:42:00,320 [INFO] subscription-manager:14440:MainThread @connection.py:552 - Response: status=500, requestUuid=7f89cca6-78f6-443a-811f-89a38cd2ff2e, request="GET /subscription/owners/6340056/pools?consumer=f614237c-e4e4-4387-b00d-d9de22bc18dc" That's a 500 on the server side. No problem with subscription-manager per se. Likewise, your most recent error corresponds to the entry 2018-01-12 15:06:40,380 [INFO] subscription-manager:4624:MainThread @connection.py:551 - Response: status=500, requestUuid=af44a6f7-76df-440a-95f9-b23024b8b371, request="POST /subscription/consumers/26192c87-971b-4184-877c-9dacff591e8f/entitlements?pool=8a85f98260c27fc50160c323263339ff" I see two problems: a) The error messages here are of very poor quality. They don't indicate that the error is on the server rather than on the client. b) There's a server error somewhere that's keeping you from getting your subscriptions. I will go ahead and correct the first issue in this bug. The second issue may require a little more investigation. Jaromir, Looking back on the log you provided, I can track that on the server-side to a "Caused by: java.sql.SQLException: Lock wait timeout exceeded; try restarting transaction" error. We see these errors occasionally (especially with the employee SKU since it contains so many products) when systems are attempted to attach to a pool. They're just an indication that there's a lot of contention for the resources of a specific pool. Alexander, While I've made some minor client corrections, I still haven't determined what was causing the issue you were seeing. Are you running subscription-manager in a container? If so, I'm afraid that's not supported. (In reply to Alex Wood from comment #7) > I see two problems: > > a) The error messages here are of very poor quality. They don't indicate that the error is on the server rather than on the client. > b) There's a server error somewhere that's keeping you from getting your subscriptions. > > I will go ahead and correct the first issue in this bug. Demonstrating a small example of the behavior change introduced by the improvement in https://github.com/subscription-manager/pull/1759 Here's a "before improvement" behavior for registering with invalid credentials... [root@jsefler-rhel7 ~]# subscription-manager register --username=foo --password=bar --serverurl=subscription.rhsm.redhat.com:443/subscription Registering to: subscription.rhsm.redhat.com:443/subscription Invalid username or password. To create a login, please visit https://www.redhat.com/wapps/ugc/register.html Here's an "after improvement" behavior for registering with invalid credentials... [root@jsefler-rhel7 ~]# subscription-manager register --username=foo --password=bar --serverurl=subscription.rhsm.redhat.com:443/subscription Registering to: subscription.rhsm.redhat.com:443/subscription HTTP error (401 - Unauthorized): Invalid username or password. To create a login, please visit https://www.redhat.com/wapps/ugc/register.html Notice the additional "HTTP error (401 - Unauthorized): " code information. I would like to move this bug to VERIFIED, but there are several details that have been raised in this bugzilla that should be put to rest.... _______________________________ First. Regarding "running subscription-manager in a container" as shown in comment 0. I agree with comment 4 that subscription-manager should be disabled inside a container since the entitlements from the host are shared with the container making it unnecessary and unsupported to run subscription-manager from within a running container. If you need access to the CDN from within a running container, you need to register the container's host system and attach subscriptions to the host that provide entitlements that you wish to utilize inside the running container. Here is the expected response from subscription-manager from within a rhel7 container. [root@hpe-dl380pgen8-02-vm-11 ~]# subscription-manager register --username=rhelentqe --auto-attach Registering to: subscription.rhsm.redhat.com:443/subscription Password: The system has been registered with ID: 6441bec2-2050-4d16-b780-9e11747f7744 The registered system name is: hpe-dl380pgen8-02-vm-11.hpe2.lab.eng.bos.redhat.com Installed Product Current Status: Product Name: Red Hat Enterprise Linux Server Status: Subscribed [root@hpe-dl380pgen8-02-vm-11 ~]# docker run -i -t --rm registry.access.redhat.com/rhel7 /bin/bash Unable to find image 'registry.access.redhat.com/rhel7:latest' locally Trying to pull repository registry.access.redhat.com/rhel7 ... latest: Pulling from registry.access.redhat.com/rhel7 ec0a4551131f: Pull complete 448f7cafed66: Pull complete Digest: sha256:b0818ebc44a7e45a4c5c839a5b63282fcc6b0ad5f92ffe316a2306a3e84d0594 Status: Downloaded newer image for registry.access.redhat.com/rhel7:latest [root@214c9dbeb1fb /]# [root@214c9dbeb1fb /]# subscription-manager list --available subscription-manager is disabled when running inside a container. Please refer to your host system for subscription management. [root@214c9dbeb1fb /]# [root@214c9dbeb1fb /]# yum repolist Loaded plugins: ovl, product-id, search-disabled-repos, subscription-manager rhel-7-server-rpms | 3.5 kB 00:00:00 (1/3): rhel-7-server-rpms/7Server/x86_64/group | 631 kB 00:00:00 (2/3): rhel-7-server-rpms/7Server/x86_64/updateinfo | 3.7 MB 00:00:01 (3/3): rhel-7-server-rpms/7Server/x86_64/primary_db | 69 MB 00:00:05 repo id repo name status rhel-7-server-rpms/7Server/x86_64 Red Hat Enterprise Linux 7 Server (RPMs) 28756 repolist: 28756 [root@214c9dbeb1fb /]# VERIFIED: This is the expected behaviour when running subscription-manager inside the latest "rhel7" container. Despite the fact that "subscription-manager is disabled when running inside a container", I can still access entitled content from the CDN through the attached entitlement shared from the host system. _______________________________ Second: Regarding the attempt to list available subscriptions from an account that appears to have an awful lot of "*Employee SKU*" subscriptions, I don't think this is a realistic customer situation. Moreover, the errors encountered in comment 0 and comment 3 appear to be the result of a server-side 500 error (as discussed in comment 7 and comment 8) which is not always reproducible but detection of it was improved with the changes demonstrated in comment 11. I will try to reproduce the situation one more time from an account with several Employee SKUs... [root@hpe-dl380pgen8-02-vm-11 ~]# subscription-manager register --username=qa Registering to: subscription.rhsm.redhat.com:443/subscription Password: The system has been registered with ID: 15fc78d0-15db-472f-895f-86bd6f91ec0b The registered system name is: hpe-dl380pgen8-02-vm-11.hpe2.lab.eng.bos.redhat.com [root@hpe-dl380pgen8-02-vm-11 ~]# [root@hpe-dl380pgen8-02-vm-11 ~]# subscription-manager list --available --matches "*Employee SKU*" | wc -l 301 [root@hpe-dl380pgen8-02-vm-11 ~]# subscription-manager list --available 1>/tmp/stdout [root@hpe-dl380pgen8-02-vm-11 ~]# echo $? 0 [root@hpe-dl380pgen8-02-vm-11 ~]# [root@hpe-dl380pgen8-02-vm-11 ~]# grep corrupted /tmp/stdout [root@hpe-dl380pgen8-02-vm-11 ~]# VERIFIED: Although my account has only 301 occurrences of "*Employee SKU*" in my list of available subscription (as opposed to 1277 in comment 0), I did not encounter a server-side error that manifested itself as "System certificates corrupted". Worksforme. _______________________________ Moving this bug to VERIFIED. If "System certificates corrupted." continues to be a problem, please open a new bugzilla. [root@hpe-dl380pgen8-02-vm-11 ~]# subscription-manager version server type: Red Hat Subscription Management subscription management server: 2.9.21-1 subscription management rules: 5.37 subscription-manager: 1.24.32 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (subscription-manager bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:3866 The needinfo request[s] on this closed bug have been removed as they have been unresolved for 500 days |