Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1389802 - upgrade to 6.2.3 failed at update_subscription_facet_backend_data (undefined method `inject' for nil:NilClass)
Summary: upgrade to 6.2.3 failed at update_subscription_facet_backend_data (undefined ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Upgrades
Version: 6.2.3
Hardware: x86_64
OS: Linux
high
medium
Target Milestone: Unspecified
Assignee: Justin Sherrill
QA Contact: Jitendra Yejare
URL:
Whiteboard:
Depends On:
Blocks: 1399395 1426397
TreeView+ depends on / blocked
 
Reported: 2016-10-28 16:56 UTC by Patrick C. F. Ernzer
Modified: 2020-04-15 14:47 UTC (History)
11 users (show)

Fixed In Version: tfm-rubygem-katello-3.0.0.115-1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1426397 (view as bug list)
Environment:
Last Closed: 2017-05-01 13:56:05 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
foreman-debug (5.87 MB, application/x-xz)
2016-10-28 16:56 UTC, Patrick C. F. Ernzer 		no flags Details
LOG: foreman-rake katello:upgrades:3.0:update_subscription_facet_backend_data (3.68 KB, text/plain)
2016-10-28 16:59 UTC, Patrick C. F. Ernzer 		no flags Details
shell output from an upgrade attempt (16.41 KB, text/plain)
2016-10-28 17:00 UTC, Patrick C. F. Ernzer 		no flags Details
View All


Links
System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 17612 0 High Closed upgrade failed at update_subscription_facet_backend_data (undefined method `inject' for nil:NilClass) 2020-11-17 19:11:49 UTC
Red Hat Product Errata RHBA-2017:1191 0 normal SHIPPED_LIVE Satellite 6.2.9 Async Bug Release 2017-05-01 17:49:42 UTC

Description Patrick C. F. Ernzer 2016-10-28 16:56:42 UTC 
Created attachment 1215047 [details]
foreman-debug

Description of problem:
upgrading my test Sat 6.2.2 to 6.2.3 failed

failure was at the
  foreman-rake katello:upgrades:3.0:update_subscription_facet_backend_data
step


Version-Release number of selected component (if applicable):
6.2.3

How reproducible:
always (re-running "foreman-rake katello:upgrades:3.0:update_subscription_facet_backend_data" or "satellite-installer --scenario satellite --upgrade" does not help)

Steps to Reproduce:
1. upgrade 6.2.2 as per https://access.redhat.com/documentation/en/red-hat-satellite/6.2/single/installation-guide#updating_satellite_server_to_next_minor_version


Actual results:
tmp/subscription_facet_upgrade-1477666836.log
Error: sattestclient04.sattest.pcfe.net - 7

undefined method `inject' for nil:NilClass
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.81/app/lib/katello/http_resource.rb:197:in `hash_to_query'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.81/app/lib/katello/resources/candlepin.rb:102:in `get'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.81/app/services/katello/candlepin/consumer.rb:17:in `block in <class:Consumer>'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.81/app/lib/katello/lazy_accessor.rb:160:in `instance_eval'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.81/app/lib/katello/lazy_accessor.rb:160:in `run_initializer'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.81/app/lib/katello/lazy_accessor.rb:139:in `lazy_attribute_get'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.81/app/lib/katello/lazy_accessor.rb:60:in `block (2 levels) in lazy_accessor'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.81/lib/katello/tasks/upgrades/3.0/update_subscription_facet_backend_data.rake:43:in `block (5 levels) in <top (required)>'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/relation/batches.rb:52:in `block (2 levels) in find_each'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/relation/batches.rb:52:in `each'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/relation/batches.rb:52:in `block in find_each'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/relation/batches.rb:125:in `find_in_batches'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/relation/batches.rb:51:in `find_each'
/opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/querying.rb:9:in `find_each'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.81/lib/katello/tasks/upgrades/3.0/update_subscription_facet_backend_data.rake:41:in `block (4 levels) in <top (required)>'
/opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/task.rb:240:in `call'
/opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/task.rb:240:in `block in execute'
/opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/task.rb:235:in `each'
/opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/task.rb:235:in `execute'
/opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/task.rb:179:in `block in invoke_with_call_chain'
/opt/rh/rh-ruby22/root/usr/share/ruby/monitor.rb:211:in `mon_synchronize'
/opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/task.rb:172:in `invoke_with_call_chain'
/opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/task.rb:165:in `invoke'
/opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/application.rb:150:in `invoke_task'
/opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/application.rb:106:in `block (2 levels) in top_level'
/opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/application.rb:106:in `each'
/opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/application.rb:106:in `block in top_level'
/opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/application.rb:115:in `run_with_threads'
/opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/application.rb:100:in `top_level'
/opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/application.rb:78:in `block in run'
/opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/application.rb:176:in `standard_exception_handling'
/opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/application.rb:75:in `run'
/opt/rh/rh-ruby22/root/usr/bin/rake:33:in `<main>'


Expected results:
successful upgrade

Additional info:
post mess I looked at errata for the Sat itself and notived that it was not visible at https://access.redhat.com/management/distributors/ (maybe I searched badly)

subscription-manager register --type=satellite --force
subscription-manager attach --pool=SOMETHING
18:50:50 [root@satellite ~]# subscription-manager repos --list-enabled
+----------------------------------------------------------+
    Available Repositories in /etc/yum.repos.d/redhat.repo
+----------------------------------------------------------+
Repo ID:   rhel-server-rhscl-7-rpms
Repo Name: Red Hat Software Collections RPMs for Red Hat Enterprise Linux 7 Server
Repo URL:  https://cdn.redhat.com/content/dist/rhel/server/7/$releasever/$basearch/rhscl/1/os
Enabled:   1

Repo ID:   rhel-7-server-rpms
Repo Name: Red Hat Enterprise Linux 7 Server (RPMs)
Repo URL:  https://cdn.redhat.com/content/dist/rhel/server/7/$releasever/$basearch/os
Enabled:   1

Repo ID:   rhel-7-server-satellite-6.2-rpms
Repo Name: Red Hat Satellite 6.2 (for RHEL 7 Server) (RPMs)
Repo URL:  https://cdn.redhat.com/content/dist/rhel/server/7/7Server/$basearch/satellite/6.2/os
Enabled:   1

18:51:07 [root@satellite ~]# 
does not help
Comment 1 Patrick C. F. Ernzer 2016-10-28 16:59:00 UTC 
Created attachment 1215049 [details]
LOG: foreman-rake katello:upgrades:3.0:update_subscription_facet_backend_data

18:57:42 [root@satellite ~]#   foreman-rake katello:upgrades:3.0:update_subscription_facet_backend_data
Updating backend data for subscription facets
***********************************
*************WARNING***************
Errors detected during upgrade step.
Details saved to: /tmp/subscription_facet_upgrade-1477673881.log
This step can be rerun with:
  foreman-rake katello:upgrades:3.0:update_subscription_facet_backend_data
You are likely encountering a bug.
***********************************
18:58:01 [root@satellite ~]#
Comment 2 Patrick C. F. Ernzer 2016-10-28 17:00:39 UTC 
Created attachment 1215050 [details]
shell output from an upgrade attempt
Comment 5 Patrick C. F. Ernzer 2016-10-28 17:39:38 UTC 
note that the defined compute resource is the libvirt hypervisor the satellite itself runs as a VM on

19:08:06 [root@satellite ~]# hammer organization list
---|----------|----------|------------
ID | NAME     | LABEL    | DESCRIPTION
---|----------|----------|------------
1  | Sat Test | Sat_Test |   
---|----------|----------|------------
19:25:16 [root@satellite ~]# 
19:25:28 [root@satellite ~]# hammer organization create -name "Sat Test 2"
Could not create the organization:
  Error: Unrecognised option '-n'

  See: 'hammer organization create --help'
19:26:58 [root@satellite ~]# hammer organization create --name "Sat Test 2"
Organization created
19:27:07 [root@satellite ~]# hammer organization delete --id 1
Could not delete the organization:
  Failed to destroy a compute libvirt haswell (Libvirt) instance jenkins.sattest.pcfe.net: wrong number of arguments (0 for 1)
19:27:24 [root@satellite ~]# hammer compute-resource list
---|-----------------|---------
ID | NAME            | PROVIDER
---|-----------------|---------
1  | libvirt haswell | Libvirt
---|-----------------|---------
19:32:38 [root@satellite ~]# 
19:35:55 [root@satellite ~]# hammer compute-resource update --organization-ids 2 --id 1
Could not update the compute resource:
  Validation failed: Taxonomy has already been taken
19:36:26 [root@satellite ~]#
Comment 7 Harshad More 2016-12-01 19:02:41 UTC 
One of the customer upgraded the satellite from 6.2.1 to 6.2.4 and faced similar error
Error was --
Upgrade completed, except for this error:

***********************************
*************WARNING***************
Errors detected during upgrade step.
Details saved to: /tmp/subscription_facet_upgrade-1480516196.log
This step can be rerun with:
  foreman-rake katello:upgrades:3.0:update_subscription_facet_backend_data
You are likely encountering a bug.
***********************************

Even after rerun facing the same error
Comment 10 Justin Sherrill 2016-12-08 15:06:36 UTC 
Note that the error message:

Updating backend data for subscription facets
***********************************
*************WARNING***************
Errors detected during upgrade step.
Details saved to: /tmp/subscription_facet_upgrade-1481068894.log
This step can be rerun with:
  foreman-rake katello:upgrades:3.0:update_subscription_facet_backend_data
You are likely encountering a bug.
***********************************

is very generic and does not mean any specific bug.  Please do not associate any cases with this bug without verifying that the traceback in the log file matches the description from this bz.
Comment 11 Justin Sherrill 2016-12-08 15:09:47 UTC 
The error "Katello::Resources::Candlepin::Consumer: 410 Gone" has nothing to do with the original report of this bz and will NOT be fixed as part of it.  See https://bugzilla.redhat.com/show_bug.cgi?id=1372720#c3 for more information.

This particular bug does seem like a simple case to workaround with a patch.  In the meantime you can resolve the issue with:

# formeman-rake console
> Katello::Host::SubscriptionFacet.where(:uuid => nil).destroy_all


I would like to reiterate, if you are not seeing the error 'undefined method inject of nil:NilClass', this will do NOTHING for you.
Comment 12 Justin Sherrill 2016-12-08 15:13:33 UTC 
Created redmine issue http://projects.theforeman.org/issues/17612 from this bug
Comment 13 Bryan Kearney 2016-12-08 17:13:32 UTC 
Upstream bug assigned to jsherril
Comment 14 Bryan Kearney 2016-12-08 17:13:38 UTC 
Upstream bug assigned to jsherril
Comment 15 Satellite Program 2017-01-06 23:12:54 UTC 
Moving this bug to POST for triage into Satellite 6 since the upstream issue http://projects.theforeman.org/issues/17612 has been resolved.
Comment 16 Satellite Program 2017-02-23 21:10:00 UTC 
Please add verifications steps for this bug to help QE verify
Comment 17 Justin Sherrill 2017-04-24 18:56:58 UTC 
To reproduce:

1.  Register a client with subscription-manager
2.  Set its UUID to nil:

foreman-rake console:

>  ::Host.find_by(:name => "foo.example.com").subscription_facet.update_attributes!(:uuid => nil)

3. foreman-rake katello:upgrades:3.0:update_subscription_facet_backend_data
Comment 18 Jitendra Yejare 2017-04-25 14:59:28 UTC 
Verified !

@ Satellite 6.2.9 snap 5


Steps:
1. Register a client with satellite.
2. Set its UUID to nil

irb(main):001:0> ::Host.find_by(:name => "0dockerclientrhel6").subscription_facet.update_attributes!(:uuid => nil)
=> true

3. foreman-rake katello:upgrades:3.0:update_subscription_facet_backend_data


Results:
The update subscription facet backed data command executed without any error stated in this bug description.

So moving this bug to verified state.
Comment 20 errata-xmlrpc 2017-05-01 13:56:05 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:1191
Note You need to log in before you can comment on or make changes to this bug.