Red Hat Bugzilla – Bug 1389802
upgrade to 6.2.3 failed at update_subscription_facet_backend_data (undefined method `inject' for nil:NilClass)
Last modified: 2017-05-01 09:56:05 EDT
Created attachment 1215047 [details] foreman-debug Description of problem: upgrading my test Sat 6.2.2 to 6.2.3 failed failure was at the foreman-rake katello:upgrades:3.0:update_subscription_facet_backend_data step Version-Release number of selected component (if applicable): 6.2.3 How reproducible: always (re-running "foreman-rake katello:upgrades:3.0:update_subscription_facet_backend_data" or "satellite-installer --scenario satellite --upgrade" does not help) Steps to Reproduce: 1. upgrade 6.2.2 as per https://access.redhat.com/documentation/en/red-hat-satellite/6.2/single/installation-guide#updating_satellite_server_to_next_minor_version Actual results: tmp/subscription_facet_upgrade-1477666836.log Error: sattestclient04.sattest.pcfe.net - 7 undefined method `inject' for nil:NilClass /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.81/app/lib/katello/http_resource.rb:197:in `hash_to_query' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.81/app/lib/katello/resources/candlepin.rb:102:in `get' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.81/app/services/katello/candlepin/consumer.rb:17:in `block in <class:Consumer>' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.81/app/lib/katello/lazy_accessor.rb:160:in `instance_eval' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.81/app/lib/katello/lazy_accessor.rb:160:in `run_initializer' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.81/app/lib/katello/lazy_accessor.rb:139:in `lazy_attribute_get' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.81/app/lib/katello/lazy_accessor.rb:60:in `block (2 levels) in lazy_accessor' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.81/lib/katello/tasks/upgrades/3.0/update_subscription_facet_backend_data.rake:43:in `block (5 levels) in <top (required)>' /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/relation/batches.rb:52:in `block (2 levels) in find_each' /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/relation/batches.rb:52:in `each' /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/relation/batches.rb:52:in `block in find_each' /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/relation/batches.rb:125:in `find_in_batches' /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/relation/batches.rb:51:in `find_each' /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/querying.rb:9:in `find_each' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.81/lib/katello/tasks/upgrades/3.0/update_subscription_facet_backend_data.rake:41:in `block (4 levels) in <top (required)>' /opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/task.rb:240:in `call' /opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/task.rb:240:in `block in execute' /opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/task.rb:235:in `each' /opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/task.rb:235:in `execute' /opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/task.rb:179:in `block in invoke_with_call_chain' /opt/rh/rh-ruby22/root/usr/share/ruby/monitor.rb:211:in `mon_synchronize' /opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/task.rb:172:in `invoke_with_call_chain' /opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/task.rb:165:in `invoke' /opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/application.rb:150:in `invoke_task' /opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/application.rb:106:in `block (2 levels) in top_level' /opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/application.rb:106:in `each' /opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/application.rb:106:in `block in top_level' /opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/application.rb:115:in `run_with_threads' /opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/application.rb:100:in `top_level' /opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/application.rb:78:in `block in run' /opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/application.rb:176:in `standard_exception_handling' /opt/rh/rh-ruby22/root/usr/share/gems/gems/rake-10.4.2/lib/rake/application.rb:75:in `run' /opt/rh/rh-ruby22/root/usr/bin/rake:33:in `<main>' Expected results: successful upgrade Additional info: post mess I looked at errata for the Sat itself and notived that it was not visible at https://access.redhat.com/management/distributors/ (maybe I searched badly) subscription-manager register --type=satellite --force subscription-manager attach --pool=SOMETHING 18:50:50 [root@satellite ~]# subscription-manager repos --list-enabled +----------------------------------------------------------+ Available Repositories in /etc/yum.repos.d/redhat.repo +----------------------------------------------------------+ Repo ID: rhel-server-rhscl-7-rpms Repo Name: Red Hat Software Collections RPMs for Red Hat Enterprise Linux 7 Server Repo URL: https://cdn.redhat.com/content/dist/rhel/server/7/$releasever/$basearch/rhscl/1/os Enabled: 1 Repo ID: rhel-7-server-rpms Repo Name: Red Hat Enterprise Linux 7 Server (RPMs) Repo URL: https://cdn.redhat.com/content/dist/rhel/server/7/$releasever/$basearch/os Enabled: 1 Repo ID: rhel-7-server-satellite-6.2-rpms Repo Name: Red Hat Satellite 6.2 (for RHEL 7 Server) (RPMs) Repo URL: https://cdn.redhat.com/content/dist/rhel/server/7/7Server/$basearch/satellite/6.2/os Enabled: 1 18:51:07 [root@satellite ~]# does not help
Created attachment 1215049 [details] LOG: foreman-rake katello:upgrades:3.0:update_subscription_facet_backend_data 18:57:42 [root@satellite ~]# foreman-rake katello:upgrades:3.0:update_subscription_facet_backend_data Updating backend data for subscription facets *********************************** *************WARNING*************** Errors detected during upgrade step. Details saved to: /tmp/subscription_facet_upgrade-1477673881.log This step can be rerun with: foreman-rake katello:upgrades:3.0:update_subscription_facet_backend_data You are likely encountering a bug. *********************************** 18:58:01 [root@satellite ~]#
Created attachment 1215050 [details] shell output from an upgrade attempt
note that the defined compute resource is the libvirt hypervisor the satellite itself runs as a VM on 19:08:06 [root@satellite ~]# hammer organization list ---|----------|----------|------------ ID | NAME | LABEL | DESCRIPTION ---|----------|----------|------------ 1 | Sat Test | Sat_Test | ---|----------|----------|------------ 19:25:16 [root@satellite ~]# 19:25:28 [root@satellite ~]# hammer organization create -name "Sat Test 2" Could not create the organization: Error: Unrecognised option '-n' See: 'hammer organization create --help' 19:26:58 [root@satellite ~]# hammer organization create --name "Sat Test 2" Organization created 19:27:07 [root@satellite ~]# hammer organization delete --id 1 Could not delete the organization: Failed to destroy a compute libvirt haswell (Libvirt) instance jenkins.sattest.pcfe.net: wrong number of arguments (0 for 1) 19:27:24 [root@satellite ~]# hammer compute-resource list ---|-----------------|--------- ID | NAME | PROVIDER ---|-----------------|--------- 1 | libvirt haswell | Libvirt ---|-----------------|--------- 19:32:38 [root@satellite ~]# 19:35:55 [root@satellite ~]# hammer compute-resource update --organization-ids 2 --id 1 Could not update the compute resource: Validation failed: Taxonomy has already been taken 19:36:26 [root@satellite ~]#
One of the customer upgraded the satellite from 6.2.1 to 6.2.4 and faced similar error Error was -- Upgrade completed, except for this error: *********************************** *************WARNING*************** Errors detected during upgrade step. Details saved to: /tmp/subscription_facet_upgrade-1480516196.log This step can be rerun with: foreman-rake katello:upgrades:3.0:update_subscription_facet_backend_data You are likely encountering a bug. *********************************** Even after rerun facing the same error
Note that the error message: Updating backend data for subscription facets *********************************** *************WARNING*************** Errors detected during upgrade step. Details saved to: /tmp/subscription_facet_upgrade-1481068894.log This step can be rerun with: foreman-rake katello:upgrades:3.0:update_subscription_facet_backend_data You are likely encountering a bug. *********************************** is very generic and does not mean any specific bug. Please do not associate any cases with this bug without verifying that the traceback in the log file matches the description from this bz.
The error "Katello::Resources::Candlepin::Consumer: 410 Gone" has nothing to do with the original report of this bz and will NOT be fixed as part of it. See https://bugzilla.redhat.com/show_bug.cgi?id=1372720#c3 for more information. This particular bug does seem like a simple case to workaround with a patch. In the meantime you can resolve the issue with: # formeman-rake console > Katello::Host::SubscriptionFacet.where(:uuid => nil).destroy_all I would like to reiterate, if you are not seeing the error 'undefined method inject of nil:NilClass', this will do NOTHING for you.
Created redmine issue http://projects.theforeman.org/issues/17612 from this bug
Upstream bug assigned to jsherril@redhat.com
Moving this bug to POST for triage into Satellite 6 since the upstream issue http://projects.theforeman.org/issues/17612 has been resolved.
Please add verifications steps for this bug to help QE verify
To reproduce: 1. Register a client with subscription-manager 2. Set its UUID to nil: foreman-rake console: > ::Host.find_by(:name => "foo.example.com").subscription_facet.update_attributes!(:uuid => nil) 3. foreman-rake katello:upgrades:3.0:update_subscription_facet_backend_data
Verified ! @ Satellite 6.2.9 snap 5 Steps: 1. Register a client with satellite. 2. Set its UUID to nil irb(main):001:0> ::Host.find_by(:name => "0dockerclientrhel6").subscription_facet.update_attributes!(:uuid => nil) => true 3. foreman-rake katello:upgrades:3.0:update_subscription_facet_backend_data Results: The update subscription facet backed data command executed without any error stated in this bug description. So moving this bug to verified state.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:1191