Description of problem: When upgrading Satellite 6.2.2 to 6.2.9, update_subscription_facet_backend_data step uses 12GB of memory on a ~450 content host Satellite. This amount seems excessive and might prompt an OOM situation even in systems with 16GB ram if not enough swap is allocated. Version-Release number of selected component (if applicable): tfm-rubygem-katello-3.0.0.124-1.el7sat.noarch How reproducible: Always, with this set of content hosts. Steps to Reproduce: 1. Launch satellite-backup --scenario satellite --upgrade Actual results: Expected results: Upgrade step 'rake' takes less memory.
It looks like the amount of imported interfaces is causing the memory to spike. I suggest retesting this issue with https://github.com/theforeman/foreman/pull/5021 in place and virtual interface facts are ignored out.
hey Shimon, Thanks for the heads up. I have not applied the full patch into the reproducer system but rather perform a database cleanup as follows : irb(main):004:0> Nic::Interface.where("identifier like ?", 'veth%').count => 0 irb(main):005:0> Nic::Interface.where("identifier like ?", 'vnet%').count => 0 irb(main):006:0> Nic::Interface.where("identifier like ?", 'macvtap%').count => 0 irb(main):007:0> Nic::Interface.where("identifier like ?", 'macvtap%') => #<ActiveRecord::Relation []> irb(main):008:0> Nic::Interface.where("identifier like ?", 'tap%').count => 4425 irb(main):009:0> Nic::Interface.where("identifier like ?", 'qbr%').count => 2398 irb(main):010:0> Nic::Interface.where("identifier like ?", 'qvb%').count => 2434 irb(main):011:0> Nic::Interface.where("identifier like ?", 'qvo%').count => 4670 irb(main):012:0> Nic::Interface.where("identifier like ?", 'qr-%').count => 0 irb(main):013:0> Nic::Interface.where("identifier like ?", 'qg-%').count => 0 irb(main):014:0> Nic::Interface.where("identifier like ?", 'vlinuxbr%').count => 0 irb(main):015:0> Nic::Interface.where("identifier like ?", 'vovsbr%').count => 0 ... Nic::Interface.where("identifier like ?", 'tap%').each { |i| i.delete } Nic::Interface.where("identifier like ?", 'qbr%').each { |i| i.delete } Nic::Interface.where("identifier like ?", 'qvb%').each { |i| i.delete } Nic::Interface.where("identifier like ?", 'qvo%').each { |i| i.delete } After performing the above cleanup the update_subscription_facet_backend_data process is now in much better shape. It only consumes ~2.5Gb RAM and finishes in a more manageable ~7min : # time foreman-rake katello:upgrades:3.0:update_subscription_facet_backend_data --trace ** Invoke katello:upgrades:3.0:update_subscription_facet_backend_data (first_time) ** Invoke environment (first_time) ** Execute environment ** Execute katello:upgrades:3.0:update_subscription_facet_backend_data Updating backend data for subscription facets real 7m27.546s user 5m51.133s sys 0m14.747s If you agree I'll close this BZ and remove the reproducer system once confirmed. Thanks all!
I guess we can close this one as dupe of https://bugzilla.redhat.com/show_bug.cgi?id=1492047 then?
Sounds right to me.
*** This bug has been marked as a duplicate of bug 1492047 ***