Description of problem: Error updating system data on the server, see /var/log/rhsm/rhsm.log for more details. Version-Release number of selected component (if applicable): subscription-manager-1.27.4-1.el8.x86_64 dnf-plugin-subscription-manager-1.27.4-1.el8.x86_64 subscription-manager-cockpit-1.27.4-1.el8.noarch python3-subscription-manager-rhsm-1.27.4-1.el8.x86_64 subscription-manager-rhsm-certificates-1.27.4-1.el8.x86_64 tfm-rubygem-katello-3.16.0-0.14.rc2.el7sat.noarch katello-3.16.0-0.3.rc1.el7sat.noarch candlepin-3.1.13-1.el7sat.noarch How reproducible: NOT always Steps to Reproduce: 1. In my automation test, sometimes traceback will happen when register RHEL-8.3.0-20200609.1 against Satellite6.8-snap-4.0: rhsm-integrate: INFO: >>>Remote Run: subscription-manager register --username=admin --password=admin rhsm-integrate: INFO: <<<Return Code: 1 rhsm-integrate: INFO: <<<Output: Registering to: ent-01-vm-02.lab.eng.nay.redhat.com:443/rhsm The system has been registered with ID: 31f6d483-6bab-41aa-a78d-d4f14702fe31 The registered system name is: kvm-03-guest20.hv2.lab.eng.bos.redhat.com Traceback (most recent call last): File "/usr/lib64/python3.6/site-packages/subscription_manager/cache.py", line 174, in update_check self._sync_with_server(uep, consumer_uuid) File "/usr/lib64/python3.6/site-packages/subscription_manager/cache.py", line 512, in _sync_with_server _combined_profile File "/usr/lib64/python3.6/site-packages/rhsm/connection.py", line 1206, in updateCombinedProfile return self.conn.request_put(method, profile) File "/usr/lib64/python3.6/site-packages/rhsm/connection.py", line 880, in request_put return self._request("PUT", method, params, headers=headers) File "/usr/lib64/python3.6/site-packages/rhsm/connection.py", line 897, in _request info=info, headers=headers, cert_key_pairs=cert_key_pairs) File "/usr/lib64/python3.6/site-packages/rhsm/connection.py", line 756, in _request self.validateResponse(result, request_type, handler) File "/usr/lib64/python3.6/site-packages/rhsm/connection.py", line 836, in validateResponse handler=handler) rhsm.connection.RemoteServerException: Server error attempting a PUT to /rhsm/consumers/31f6d483-6bab-41aa-a78d-d4f14702fe31/profiles returned status 502 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/sbin/subscription-manager", line 11, in <module> load_entry_point('subscription-manager==1.27.4', 'console_scripts', 'subscription-manager')() File "/usr/lib64/python3.6/site-packages/subscription_manager/scripts/subscription_manager.py", line 87, in main return managercli.ManagerCLI().main() File "/usr/lib64/python3.6/site-packages/subscription_manager/managercli.py", line 3087, in main ret = CLI.main(self) File "/usr/lib64/python3.6/site-packages/subscription_manager/cli.py", line 183, in main return cmd.main() File "/usr/lib64/python3.6/site-packages/subscription_manager/managercli.py", line 545, in main return_code = self._do_command() File "/usr/lib64/python3.6/site-packages/subscription_manager/managercli.py", line 1491, in _do_command profile_mgr.update_check(self.cp, consumer['uuid'], True) File "/usr/lib64/python3.6/site-packages/subscription_manager/cache.py", line 474, in update_check return CacheManager.update_check(self, uep, consumer_uuid, force) File "/usr/lib64/python3.6/site-packages/subscription_manager/cache.py", line 184, in update_check raise Exception(_("Error updating system data on the server, see /var/log/rhsm/rhsm.log " Exception: Error updating system data on the server, see /var/log/rhsm/rhsm.log for more details. 2. On Satellite server, get a sosreport: [root@ent-01-vm-02 ~]# sosreport sosreport (version 3.7) This command will collect diagnostic and configuration information from this Red Hat Enterprise Linux system and installed applications. An archive containing the collected information will be generated in /var/tmp/sos.r1kPb_ and may be provided to a Red Hat support representative. Any information provided to Red Hat will be treated in accordance with the published support policies at: https://access.redhat.com/support/ The generated archive may contain data considered sensitive and its content should be reviewed by the originating organization before being passed to any third party. No changes will be made to system configuration. Press ENTER to continue, or CTRL-C to quit. Please enter the case id that you are generating this report for []: Setting up archive ... Setting up plugins ... Running plugins. Please wait ... Finishing plugins [Running: foreman] ] Finished running plugins Creating compressed archive... Your sosreport has been generated and saved in: /var/tmp/sosreport-ent-01-vm-02-2020-06-16-tmyeuhe.tar.xz The checksum is: e5eaffcbcd6991563f858b12c0a242d3 Please send this file to your support representative. Actual results: As step 1, a traceback happens sometimes. Expected results: Registration RHEL-8.3.0-20200609.1 against Satellite6.8-snap-4.0 should succeed. Additional info: Please refer attachment 'rhsm.log' and 'sosreport'
Created attachment 1697731 [details] sosreport
Created attachment 1697732 [details] rhsm.log
Hello Brad, we don't meet this bug when testing RHEL7.9&RHEL8.3 RHSM integration against Satellite6.7. And this bug can not be 100% reproduced on Satellite6.8.
This bug is NOT reproduced when registering RHEL-7.9-20200623.0 against Satellite6.8-snap-6.0.
This bug is NOT reproduced when registering RHEL-8.3.0-20200701.2 against Satellite6.8-snap-6.0.
Based on comment 5 and comment 6 close this bug as works for me . Please reopen the bug when you observe the error next time. Also moving to "subscriptions" component as the error appears to coming from the profiles end point. thanks, Rehana
This bug happens again on RHEL-8.3.0-20200811.0 against Satellite6.8.0-snap-11.0, and this bug is not reproduced 100%: 1. subscription-manager-1.27.13-1.el8.x86_64 dnf-plugin-subscription-manager-1.27.13-1.el8.x86_64 subscription-manager-cockpit-1.27.13-1.el8.noarch python3-subscription-manager-rhsm-1.27.13-1.el8.x86_64 subscription-manager-rhsm-certificates-1.27.13-1.el8.x86_64 tfm-rubygem-katello-3.16.0-0.16.rc5.1.el7sat.noarch katello-3.16.0-0.4.rc4.el7sat.noarch candlepin-3.1.16-1.el7sat.noarch 2. rhsm-integrate: INFO: Command run on: kvm-04-guest24.hv2.lab.eng.bos.redhat.com, purpose: register rhsm-integrate: INFO: >>>Remote Run: subscription-manager register --username=admin --password=admin rhsm-integrate: INFO: <<<Return Code: 70 rhsm-integrate: INFO: <<<Output: Registering to: ent-01-vm-01.lab.eng.nay.redhat.com:443/rhsm The system has been registered with ID: 7d8c8187-374e-480f-b4f4-493680d266c6 The registered system name is: kvm-04-guest24.hv2.lab.eng.bos.redhat.com Error updating system data on the server, see /var/log/rhsm/rhsm.log for more details. rhsm-integrate: ERROR: Test Failed - ERROR Message:Test Failed - Failed to register.
(In reply to qianzhan from comment #9) > This bug happens again on RHEL-8.3.0-20200811.0 against > Satellite6.8.0-snap-11.0, and this bug is not reproduced 100%: > > > 1. > subscription-manager-1.27.13-1.el8.x86_64 > dnf-plugin-subscription-manager-1.27.13-1.el8.x86_64 > subscription-manager-cockpit-1.27.13-1.el8.noarch > python3-subscription-manager-rhsm-1.27.13-1.el8.x86_64 > subscription-manager-rhsm-certificates-1.27.13-1.el8.x86_64 > tfm-rubygem-katello-3.16.0-0.16.rc5.1.el7sat.noarch > katello-3.16.0-0.4.rc4.el7sat.noarch > candlepin-3.1.16-1.el7sat.noarch > > 2. > > rhsm-integrate: INFO: Command run on: > kvm-04-guest24.hv2.lab.eng.bos.redhat.com, purpose: register > rhsm-integrate: INFO: >>>Remote Run: subscription-manager register > --username=admin --password=admin > rhsm-integrate: INFO: <<<Return Code: 70 > rhsm-integrate: INFO: <<<Output: > Registering to: ent-01-vm-01.lab.eng.nay.redhat.com:443/rhsm > The system has been registered with ID: 7d8c8187-374e-480f-b4f4-493680d266c6 > The registered system name is: kvm-04-guest24.hv2.lab.eng.bos.redhat.com > Error updating system data on the server, see /var/log/rhsm/rhsm.log for > more details. > > rhsm-integrate: ERROR: Test Failed - ERROR Message:Test Failed - Failed to > register. The attachment 'reopen_bug_rhsm.log' is the /var/log/rhsm/rhsm.log on client.
Created attachment 1711155 [details] reopen_bug_rhsm.log
Hello Qianqian, I took a look into your original sosreport and rhsm.log. What I observed was that the /var/log/foreman/production.log on the Satellite didn't contain any such request for PUT /rhsm/consumers/31f6d483-6bab-41aa-a78d-d4f14702fe31/profiles. Considering that the response code was a 502 I am wondering if there is an HTTP Proxy or other kind of server between your client and the Satellite? I think that 502 came from some server other than the Satellite.
Created attachment 1712137 [details] rhsm-2020-08-21.log
Created attachment 1712144 [details] sosreport-ent-01-vm-02-2020-08-21-jnnoljq.tar.xz
When I test RHEL-7.9-20200821.0 against Satellite-6.8.0-snap-12.0 today, I met a similar registration error on RHEL-7.9-20200821.0: 2020-08-24 23:58:34,427 [INFO]: >>>Remote Run: subscription-manager register --username=admin --password=admin --org=Default_Organization 2020-08-24 23:59:53,922 [INFO]: <<<Return Code: 70 2020-08-24 23:59:53,922 [INFO]: <<<Output: Registering to: ent-01-vm-02.lab.eng.nay.redhat.com:443/rhsm Remote server error. Please check the connection details, or see /var/log/rhsm/rhsm.log for more information. 2020-08-24 23:59:53,922 [ERROR]: Test Failed - ERROR Message:Test Failed - failed to verify that sam support registration without environments 1. I set the satellite as Comment 20. 2. I found the similar error in rhsm.log: 2020-08-25 07:03:18,706 [ERROR] subscription-manager:25065:MainThread @connection.py:643 - Response: 502 2020-08-25 07:03:18,706 [ERROR] subscription-manager:25065:MainThread @connection.py:644 - JSON parsing error: No JSON object could be decoded 2020-08-25 07:03:18,706 [ERROR] subscription-manager:25065:MainThread @cache.py:181 - Error updating system data on the server 2020-08-25 07:03:18,706 [ERROR] subscription-manager:25065:MainThread @cache.py:182 - Server error attempting a PUT to /rhsm/consumers/9ff3938d-f60b-4930-bd42-a58fda40cc3a/profiles returned status 502 Traceback (most recent call last): File "/usr/lib64/python2.7/site-packages/subscription_manager/cache.py", line 173, in update_check self._sync_with_server(uep, consumer_uuid) File "/usr/lib64/python2.7/site-packages/subscription_manager/cache.py", line 511, in _sync_with_server _combined_profile File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 1187, in updateCombinedProfile return self.conn.request_put(method, profile) File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 735, in request_put return self._request("PUT", method, params, headers=headers) File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 752, in _request info=info, headers=headers) File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 627, in _request self.validateResponse(result, request_type, handler) File "/usr/lib64/python2.7/site-packages/rhsm/connection.py", line 695, in validateResponse handler=handler) RemoteServerException: Server error attempting a PUT to /rhsm/consumers/9ff3938d-f60b-4930-bd42-a58fda40cc3a/profiles returned status 502 2020-08-25 07:03:18,708 [WARNING] subscription-manager:25065:MainThread @base_action_client.py:72 - Exception caught while running <subscription_manager.packageprofilelib.PackageProfileActionInvoker object at 0x7fd9a89321d0> update 2020-08-25 07:03:18,708 [ERROR] subscription-manager:25065:MainThread @base_action_client.py:73 - Error updating system data on the server, see /var/log/rhsm/rhsm.log for more details. Traceback (most recent call last): File "/usr/lib64/python2.7/site-packages/subscription_manager/base_action_client.py", line 63, in _run_update update_report = lib.update() File "/usr/lib64/python2.7/site-packages/subscription_manager/certlib.py", line 32, in update self.report = self.locker.run(self._do_update) File "/usr/lib64/python2.7/site-packages/subscription_manager/certlib.py", line 18, in run return action() File "/usr/lib64/python2.7/site-packages/subscription_manager/packageprofilelib.py", line 26, in _do_update return action.perform() File "/usr/lib64/python2.7/site-packages/subscription_manager/packageprofilelib.py", line 42, in perform ret = profile_mgr.update_check(self.uep, consumer_identity.uuid, force=force_upload) File "/usr/lib64/python2.7/site-packages/subscription_manager/cache.py", line 473, in update_check return CacheManager.update_check(self, uep, consumer_uuid, force) File "/usr/lib64/python2.7/site-packages/subscription_manager/cache.py", line 183, in update_check raise Exception(_("Error updating system data on the server, see /var/log/rhsm/rhsm.log " Exception: Error updating system data on the server, see /var/log/rhsm/rhsm.log for more details. 3. sub-man version on RHEL-7.9-20200821.0: subscription-manager-1.24.42-1.el7.x86_64 4. Please refer attachments: rhel7.9-rc1.3.rhsmlog satellite_puma_error.log satellite_puma.log sosreport-ent-01-vm-02-2020-08-25-ovwgumx.tar.xz
Created attachment 1712504 [details] rhel7.9-rc1.3.rhsmlog
Created attachment 1712505 [details] satellite_puma_error.log
Created attachment 1712506 [details] satellite_puma.log
Moving back ON_QA for a re-test now that we are running Passenger again.
As per comment 41 and comment 42, move this bug from ON_QA to VERIFIED.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: Satellite 6.8 release), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:4366