Bugzilla (bugzilla.redhat.com) will be under maintenance for infrastructure upgrades and will not be unavailable on July 31st between 12:30 AM - 05:30 AM UTC. We appreciate your understanding and patience. You can follow status.redhat.com for details.
Bug 1348135 - [upgrade] Exceptions in production.log while fetching a candlepin consumer using 'GET' request
Summary: [upgrade] Exceptions in production.log while fetching a candlepin consumer us...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Upgrades
Version: 6.2.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: Unspecified
Assignee: satellite6-bugs
QA Contact: Katello QA List
URL:
Whiteboard:
Depends On:
Blocks: 1335807
TreeView+ depends on / blocked
 
Reported: 2016-06-20 09:04 UTC by Sachin Ghai
Modified: 2019-09-26 14:01 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Release Note
Doc Text:
During a Satellite 6.2 upgrade, as part of the host unification feature, if a host and content host are found with the same name but within different organizations, the content host will be unregistered. This will remove the content host from the Satellite. Consequently, once the content host has been unregistered by the upgrade process, it is possible that errors will be observed in /var/log/foreman/production.log as the content host may still attempt to interact with the Satellite via Red Hat Subscription Manager (RHSM). The errors can include ones similar to the following: 2016-06-20 04:02:30 [app] [E] RestClient::Unauthorized: Katello::Resources::Candlepin::Consumer: 401 Unauthorized {"displayMessage":"Invalid oauth unit or secret","requestUuid":"56562131-15b1-482b-86c3-a74da4b19491"} (GET /candlepin/consumers/c1d5c956-b9c5-4088-914b-fd5e0e828c32) 2016-06-20 04:22:23 [app] [E] RestClient::Gone: Katello::Resources::Candlepin::Consumer: 410 Gone {"displayMessage":"Unit c1d5c956-b9c5-4088-914b-fd5e0e828c32 has been deleted","requestUuid":"4d433cb8-e94a-49dc-9d2f-eebdc7066e55","deletedId":"c1d5c956-b9c5-4088-914b-fd5e0e828c32"} Workaround Prior to performing the Satellite 6.2 upgrade, the user should execute the pre-upgrade script on the Satellite 6.1 Server to obtain a summary of the content hosts that will be unregistered. The script may be executed as: foreman-rake katello:upgrade_check Prior to the upgrade, the user should then update the hosts and content hosts to ensure that they are in the correct organizations. If this is not done prior to the upgrade, the user will need to re-register each of those content hosts using the Red Hat Subscription Manager command-line tool after the upgrade has completed.
Clone Of:
Environment:
Last Closed: 2016-07-27 11:31:43 UTC
Target Upstream Version:


Attachments (Terms of Use)
foreman-debug (1.06 MB, application/x-xz)
2016-06-20 09:11 UTC, Sachin Ghai
no flags Details

Description Sachin Ghai 2016-06-20 09:04:13 UTC
Description of problem:

I found following exceptions in logs while performing upgrade from sat6.1.9 -> sat6.2 GA snap16 on rhel7:


==> /var/log/foreman/production.log <==
2016-06-20 04:02:29 [app] [I] Started GET "/rhsm/consumers/c1d5c956-b9c5-4088-914b-fd5e0e828c32" for 10.8.193.1 at 2016-06-20 04:02:29 -0400
2016-06-20 04:02:29 [app] [I] Processing by Katello::Api::Rhsm::CandlepinProxiesController#consumer_show as JSON
2016-06-20 04:02:29 [app] [I]   Parameters: {"id"=>"c1d5c956-b9c5-4088-914b-fd5e0e828c32"}

==> /var/log/foreman/production.log <==
2016-06-20 04:02:30 [app] [E] RestClient::Unauthorized: Katello::Resources::Candlepin::Consumer: 401 Unauthorized {"displayMessage":"Invalid oauth unit or secret","requestUuid":"56562131-15b1-482b-86c3-a74da4b19491"} (GET /candlepin/consumers/c1d5c956-b9c5-4088-914b-fd5e0e828c32)
 | Body: {"displayMessage":"Invalid oauth unit or secret","requestUuid":"56562131-15b1-482b-86c3-a74da4b19491"}
 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/abstract_response.rb:48:in `return!'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:230:in `process_result'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/rbovirt-0.0.37/lib/restclient_ext/request.rb:50:in `block in transmit'
 | /opt/rh/rh-ruby22/root/usr/share/ruby/net/http.rb:853:in `start'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/rbovirt-0.0.37/lib/restclient_ext/request.rb:44:in `transmit'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:64:in `execute'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:33:in `execute'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/resource.rb:51:in `get'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.49/app/lib/katello/http_resource.rb:84:in `get'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.49/app/lib/katello/resources/candlepin.rb:100:in `get'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.49/app/controllers/katello/api/rhsm/candlepin_proxies_controller.rb:277:in `block in find_host'


Version-Release number of selected component (if applicable):
upgrade from sat6.1.9 -> sat6.2 GA snap16 on rhel7:

How reproducible:


Steps to Reproduce:
1. set sat6.2 repo
2. katello-service stop
3. yum update -y
4. satellite-installer --scenario satellite --upgrade

Actual results:
==> /var/log/foreman/production.log <==
2016-06-20 04:02:30 [app] [E] RestClient::Unauthorized: Katello::Resources::Candlepin::Consumer: 401 Unauthorized {"displayMessage":"Invalid oauth unit or secret","requestUuid":"56562131-15b1-482b-86c3-a74da4b19491"} (GET /candlepin/consumers/c1d5c956-b9c5-4088-914b-fd5e0e828c32)
 | Body: {"displayMessage":"Invalid oauth unit or secret","requestUuid":"56562131-15b1-482b-86c3-a74da4b19491"}
 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/abstract_response.rb:48:in `return!'


Later, following exception keep on throwing in production.log:
==> /var/log/foreman/production.log <==
2016-06-20 04:22:23 [app] [I] Started GET "/rhsm/consumers/c1d5c956-b9c5-4088-914b-fd5e0e828c32" for 10.8.193.1 at 2016-06-20 04:22:23 -0400
2016-06-20 04:22:23 [app] [I] Processing by Katello::Api::Rhsm::CandlepinProxiesController#consumer_show as JSON
2016-06-20 04:22:23 [app] [I]   Parameters: {"id"=>"c1d5c956-b9c5-4088-914b-fd5e0e828c32"}

==> /var/log/foreman/production.log <==
2016-06-20 04:22:23 [app] [E] RestClient::Gone: Katello::Resources::Candlepin::Consumer: 410 Gone {"displayMessage":"Unit c1d5c956-b9c5-4088-914b-fd5e0e828c32 has been deleted","requestUuid":"4d433cb8-e94a-49dc-9d2f-eebdc7066e55","deletedId":"c1d5c956-b9c5-4088-914b-fd5e0e828c32"} (GET /candlepin/consumers/c1d5c956-b9c5-4088-914b-fd5e0e828c32)
 | Body: {"displayMessage":"Unit c1d5c956-b9c5-4088-914b-fd5e0e828c32 has been deleted","requestUuid":"4d433cb8-e94a-49dc-9d2f-eebdc7066e55","deletedId":"c1d5c956-b9c5-4088-914b-fd5e0e828c32"}
 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/abstract_response.rb:48:in `return!'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:230:in `process_result'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/rbovirt-0.0.37/lib/restclient_ext/request.rb:50:in `block in transmit'
 |


Expected results:
No exception should be thrown in production.log while upgrade is on going.

Additional info:

Comment 1 Sachin Ghai 2016-06-20 09:05:23 UTC
Please note that this is a new issue and we never got such exception with earlier snaps during upgrade.

Comment 2 Sachin Ghai 2016-06-20 09:07:19 UTC
Also, as per logs, we got the reported issue with "Upgrade_step: migrate_foreman", through upgrade was completed successfully. I don't see any issue/error on stdout.

Comment 3 Sachin Ghai 2016-06-20 09:11:40 UTC
Created attachment 1169795 [details]
foreman-debug

Comment 4 Sachin Ghai 2016-06-20 10:55:29 UTC
I guess this consumer is our external capsule. When satellite upgrade is completed and I re-regsitered the capsule with satellite, that exception was completely gone.

Comment 6 Brad Buckingham 2016-06-20 16:14:18 UTC
Hi Sachin,  this bug falls in to a similar category as bug 1344215.  These type of errors can be observed when a host has been unregistered from the Satellite server side.  This is another one that we should document in release notes.

Comment 8 Sachin Ghai 2016-06-21 11:02:27 UTC
Hey Brad: thanks for confirming the issue. But this exception is ugly as I saw this exception a bunch of times while performing the upgrade. Since same reason as mentioned here https://bugzilla.redhat.com/show_bug.cgi?id=1344215#c6 applies to this issue. So let's document in release notes.

Comment 10 Bryan Kearney 2016-07-27 11:31:43 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1501


Note You need to log in before you can comment on or make changes to this bug.