Created attachment 1344304 [details]
rhsm.log from subscription attempts.

I am experiencing the very same issue, also with the Employee SKU, except for me, the subscription-manager also randomly throws in a complaint object serialization:

  [jhradilek@server ~]$ sudo subscription-manager register
  Registering to: subscription.rhsm.redhat.com:443/subscription
  Username: jhradile
  Password: 
  The system has been registered with ID: cc3629b3-ede3-481a-b269-ba63a501d6cb 
  [jhradilek@server ~]$ sudo subscription-manager list --available
  Unable to serialize objects to JSON.
  [jhradilek@server ~]$ sudo subscription-manager list --available
  System certificates corrupted. Please reregister.

That this is a fresh installation of RHEL 7.4 Server installed using the rhel-server-7.4-x86_64-dvd.iso image downloaded from the Customer Portal. Please note that the command I used is actually recommended by our official Product Documentation:

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system_administrators_guide/chap-subscription_and_support-registering_a_system_and_managing_subscriptions#sect-Subscription_and_Support-Registering_a_System_and_Managing_Subscriptions-Registering_the_System_and_Attaching_Subscriptions

I want to hope that our customers are not affected by this. Has anybody looked into this yet?

Alexander, thank you for your workaround! It didn't occur to me to try using --matches to limit the number of entries, and it indeed solves the problem.

Jaromir,

Are you running subscription-manager in a container?  From Alexander's comments it looks like he was running subscription-manager in a container which isn't really supported (since subscription-manager uses DBus and a Docker container won't have a DBus daemon running).

If you are running outside a container, would you mind reproducing the issue and then attaching the /var/log/rhsm/rhsm.log file?  I've tried to reproduce this issue with my own account that has an Employee SKU entitlement, but didn't encounter the problem.

Hi Alex,

Thank you for looking into this. I was not running subscription-manager in a container, but a fresh installation of RHEL 7.4 Server running in a virtual machine for testing purposes (in QEMU/KVM to be precise).

I tried to reproduce the error today in the same virtual machine and was prompted to reregister the system by using the following two commands:

  sudo subscription-manager clean
  sudo subscription-manager register

After doing so, I can get the complete list of available subscriptions without running into the error I reported. This didn't work in November. I am, however, still not able to attach the subscription I want, but that might be a different problem:

  [jhradilek@server ~]$ subscription-manager attach --pool 8a85f98260c27fc50160c323263339ff
  You are attempting to run "subscription-manager" which requires administrative
privileges, but more information is needed in order to do so.
  Authenticating as "root"
  Password:
  Runtime Error could not extract ResultSet at com.mysql.jdbc.SQLError.createSQLException:1,078

I am going to attach my /var/log/rhsm/rhsm.log file which includes records from both today and November.

Created attachment 1380434 [details]
The /var/log/rhsm/rhsm.log file

Jaromir,

For your problem in November, I see the following in the log file:

2017-11-21 20:42:00,320 [INFO] subscription-manager:14440:MainThread @connection.py:552 - Response: status=500, requestUuid=7f89cca6-78f6-443a-811f-89a38cd2ff2e, request="GET /subscription/owners/6340056/pools?consumer=f614237c-e4e4-4387-b00d-d9de22bc18dc"

That's a 500 on the server side.  No problem with subscription-manager per se.

Likewise, your most recent error corresponds to the entry

2018-01-12 15:06:40,380 [INFO] subscription-manager:4624:MainThread @connection.py:551 - Response: status=500, requestUuid=af44a6f7-76df-440a-95f9-b23024b8b371, request="POST /subscription/consumers/26192c87-971b-4184-877c-9dacff591e8f/entitlements?pool=8a85f98260c27fc50160c323263339ff"

I see two problems:

a) The error messages here are of very poor quality.  They don't indicate that the error is on the server rather than on the client.
b) There's a server error somewhere that's keeping you from getting your subscriptions.

I will go ahead and correct the first issue in this bug.  The second issue may require a little more investigation.

Jaromir,

Looking back on the log you provided, I can track that on the server-side to a "Caused by: java.sql.SQLException: Lock wait timeout exceeded; try restarting transaction" error.  We see these errors occasionally (especially with the employee SKU since it contains so many products) when systems are attempted to attach to a pool.  They're just an indication that there's a lot of contention for the resources of a specific pool.

Alexander,

While I've made some minor client corrections, I still haven't determined what was causing the issue you were seeing.  Are you running subscription-manager in a container?  If so, I'm afraid that's not supported.

(In reply to Alex Wood from comment #7)
> I see two problems:
> 
> a) The error messages here are of very poor quality.  They don't indicate that the error is on the server rather than on the client.
> b) There's a server error somewhere that's keeping you from getting your subscriptions.
> 
> I will go ahead and correct the first issue in this bug.


Demonstrating a small example of the behavior change introduced by the improvement in https://github.com/subscription-manager/pull/1759


Here's a "before improvement" behavior for registering with invalid credentials...

[root@jsefler-rhel7 ~]# subscription-manager register --username=foo --password=bar --serverurl=subscription.rhsm.redhat.com:443/subscription
Registering to: subscription.rhsm.redhat.com:443/subscription
Invalid username or password. To create a login, please visit https://www.redhat.com/wapps/ugc/register.html


Here's an "after improvement" behavior for registering with invalid credentials...

[root@jsefler-rhel7 ~]# subscription-manager register --username=foo --password=bar --serverurl=subscription.rhsm.redhat.com:443/subscription
Registering to: subscription.rhsm.redhat.com:443/subscription
HTTP error (401 - Unauthorized): Invalid username or password. To create a login, please visit https://www.redhat.com/wapps/ugc/register.html


Notice the additional "HTTP error (401 - Unauthorized): " code information.

I would like to move this bug to VERIFIED, but there are several details that have been raised in this bugzilla that should be put to rest....

_______________________________
First. Regarding "running subscription-manager in a container" as shown in comment 0.  I agree with comment 4 that subscription-manager should be disabled inside a container since the entitlements from the host are shared with the container making it unnecessary and unsupported to run subscription-manager from within a running container.  If you need access to the CDN from within a running container, you need to register the container's host system and attach subscriptions to the host that provide entitlements that you wish to utilize inside the running container.  Here is the expected response from subscription-manager from within a rhel7 container.


[root@hpe-dl380pgen8-02-vm-11 ~]# subscription-manager register --username=rhelentqe --auto-attach
Registering to: subscription.rhsm.redhat.com:443/subscription
Password: 
The system has been registered with ID: 6441bec2-2050-4d16-b780-9e11747f7744
The registered system name is: hpe-dl380pgen8-02-vm-11.hpe2.lab.eng.bos.redhat.com
Installed Product Current Status:
Product Name: Red Hat Enterprise Linux Server
Status:       Subscribed

[root@hpe-dl380pgen8-02-vm-11 ~]# docker run -i -t --rm registry.access.redhat.com/rhel7 /bin/bash 
Unable to find image 'registry.access.redhat.com/rhel7:latest' locally
Trying to pull repository registry.access.redhat.com/rhel7 ... 
latest: Pulling from registry.access.redhat.com/rhel7
ec0a4551131f: Pull complete 
448f7cafed66: Pull complete 
Digest: sha256:b0818ebc44a7e45a4c5c839a5b63282fcc6b0ad5f92ffe316a2306a3e84d0594
Status: Downloaded newer image for registry.access.redhat.com/rhel7:latest
[root@214c9dbeb1fb /]# 
[root@214c9dbeb1fb /]# subscription-manager list --available
subscription-manager is disabled when running inside a container. Please refer to your host system for subscription management.

[root@214c9dbeb1fb /]# 
[root@214c9dbeb1fb /]# yum repolist          
Loaded plugins: ovl, product-id, search-disabled-repos, subscription-manager
rhel-7-server-rpms                                                             | 3.5 kB  00:00:00     
(1/3): rhel-7-server-rpms/7Server/x86_64/group                                 | 631 kB  00:00:00     
(2/3): rhel-7-server-rpms/7Server/x86_64/updateinfo                            | 3.7 MB  00:00:01     
(3/3): rhel-7-server-rpms/7Server/x86_64/primary_db                            |  69 MB  00:00:05     
repo id                                     repo name                                           status
rhel-7-server-rpms/7Server/x86_64           Red Hat Enterprise Linux 7 Server (RPMs)            28756
repolist: 28756
[root@214c9dbeb1fb /]# 

VERIFIED: This is the expected behaviour when running subscription-manager inside the latest "rhel7" container.  Despite the fact that "subscription-manager is disabled when running inside a container", I can still access entitled content from the CDN through the attached entitlement shared from the host system.

_______________________________
Second: Regarding the attempt to list available subscriptions from an account that appears to have an awful lot of "*Employee SKU*" subscriptions, I don't think this is a realistic customer situation.  Moreover, the errors encountered in comment 0 and comment 3 appear to be the result of a server-side 500 error (as discussed in comment 7 and comment 8) which is not always reproducible but detection of it was improved with the changes demonstrated in comment 11.  I will try to reproduce the situation one more time from an account with several Employee SKUs...

[root@hpe-dl380pgen8-02-vm-11 ~]# subscription-manager register --username=qa
Registering to: subscription.rhsm.redhat.com:443/subscription
Password: 
The system has been registered with ID: 15fc78d0-15db-472f-895f-86bd6f91ec0b
The registered system name is: hpe-dl380pgen8-02-vm-11.hpe2.lab.eng.bos.redhat.com
[root@hpe-dl380pgen8-02-vm-11 ~]# 
[root@hpe-dl380pgen8-02-vm-11 ~]# subscription-manager list --available --matches "*Employee SKU*" | wc -l
301
[root@hpe-dl380pgen8-02-vm-11 ~]# subscription-manager list --available 1>/tmp/stdout
[root@hpe-dl380pgen8-02-vm-11 ~]# echo $?
0
[root@hpe-dl380pgen8-02-vm-11 ~]# 
[root@hpe-dl380pgen8-02-vm-11 ~]# grep corrupted /tmp/stdout
[root@hpe-dl380pgen8-02-vm-11 ~]# 

VERIFIED: Although my account has only 301 occurrences of "*Employee SKU*" in my list of available subscription (as opposed to 1277 in comment 0), I did not encounter a server-side error that manifested itself as "System certificates corrupted".  Worksforme.

_______________________________

Moving this bug to VERIFIED.  If "System certificates corrupted." continues to be a problem, please open a new bugzilla.

[root@hpe-dl380pgen8-02-vm-11 ~]# subscription-manager version
server type: Red Hat Subscription Management
subscription management server: 2.9.21-1
subscription management rules: 5.37
subscription-manager: 1.24.32

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (subscription-manager bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:3866

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 500 days