Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 906902 - reindexing elasticsearch fails when too many packages
Summary: reindexing elasticsearch fails when too many packages
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: WebUI
Version: Nightly
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: Unspecified
Assignee: Justin Sherrill
QA Contact: Kedar Bidarkar
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-02-01 20:46 UTC by Tom McKay
Modified: 2019-09-26 14:06 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-09-11 12:20:54 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 786572 0 unspecified CLOSED elasticsearch can get killed due to out of memory errors. 2021-02-22 00:41:40 UTC

Internal Links: 786572

Description Tom McKay 2013-02-01 20:46:55 UTC 
I have sync'ed and promoted three OSes: RHEL, f17, raspberry pi f17

Running 'rake reindex' times out after blowing up httpd to 8+ gig memory.

It was suggested that it would be better to "chunk" back the repository packages a couple hundred at a time so as not to overwhelm the system.

Perhaps fix for https://bugzilla.redhat.com/show_bug.cgi?id=786572 not adequate (or different problem altogether)
Comment 2 Justin Sherrill 2013-05-07 17:33:21 UTC 
This should be resolved with the latest pulp and our changes we made a couple months ago.  Moving to ON_QA
Comment 4 Kedar Bidarkar 2014-08-18 15:46:29 UTC 
Tested with a box that has
a) 8GB RAM
b) sat6-GA-SNAP4
C) multiple content synced, rhel6.4,6.5, kickstart 6.5, 5.9, 5.10 repos

So with many repos synced and promoted. The below reindex was successful on sat6 with 8GB RAM.

[root@xxxx foreman]# foreman-rake katello:reindex
Elasticsearch Indices cleared.
Re-indexing User
Re-indexing Organization
Re-indexing Katello::Provider
Re-indexing Katello::Repository
Re-indexing Katello::ContentView
Re-indexing Katello::Distributor
Re-indexing Katello::ContentViewHistory
Re-indexing Katello::TaskStatus
Re-indexing Katello::ContentViewErratumFilterRule
Re-indexing Katello::ContentViewPackageFilterRule
Re-indexing Katello::ContentViewPuppetModule
Re-indexing Katello::ActivationKey
Re-indexing Katello::ContentViewPackageGroupFilterRule
Re-indexing Katello::ContentViewPuppetEnvironment
Re-indexing Katello::HostCollection
Re-indexing Katello::SyncPlan
Re-indexing Katello::Notice
Re-indexing Katello::Product
Re-indexing Katello::GpgKey
Re-indexing Katello::ContentViewFilter
Re-indexing Katello::System
Re-indexing Katello::KTEnvironment
Re-indexing Katello::Job
Re-indexing Katello::PulpTaskStatus
Re-indexing Katello::PulpSyncStatus
Re-indexing Katello::MarketingProduct
Re-indexing Katello::ContentViewErratumFilter
Re-indexing Katello::ContentViewPackageFilter
Re-indexing Katello::ContentViewPackageGroupFilter
Re-indexing Katello::Hypervisor
Re-indexing Katello::Package
Re-indexing Katello::Errata
Re-indexing Katello::PuppetModule
Re-indexing Katello::Distribution
Re-indexing Katello::PackageGroup
Re-indexing Pools


Installed Packages

    candlepin-0.9.19-1.el6_5.noarch
    candlepin-scl-1-5.el6_4.noarch
    candlepin-scl-quartz-2.1.5-5.el6_4.noarch
    candlepin-scl-rhino-1.7R3-1.el6_4.noarch
    candlepin-scl-runtime-1-5.el6_4.noarch
    candlepin-selinux-0.9.19-1.el6_5.noarch
    candlepin-tomcat6-0.9.19-1.el6_5.noarch
    createrepo-0.9.9-21.2.pulp.el6sat.noarch
    elasticsearch-0.90.10-4.el6sat.noarch
    katello-1.5.0-28.el6sat.noarch
    katello-ca-1.0-1.noarch
    katello-certs-tools-1.5.6-1.el6sat.noarch
    katello-installer-0.0.57-1.el6sat.noarch
    m2crypto-0.21.1.pulp-10.el6sat.x86_64
    mod_wsgi-3.4-1.pulp.el6sat.x86_64
    pulp-katello-0.3-3.el6sat.noarch
    pulp-nodes-common-2.4.0-0.30.beta.el6sat.noarch
    pulp-nodes-parent-2.4.0-0.30.beta.el6sat.noarch
    pulp-puppet-plugins-2.4.0-0.30.beta.el6sat.noarch
    pulp-puppet-tools-2.4.0-0.30.beta.el6sat.noarch
    pulp-rpm-plugins-2.4.0-0.30.beta.el6sat.noarch
    pulp-selinux-2.4.0-0.30.beta.el6sat.noarch
    pulp-server-2.4.0-0.30.beta.el6sat.noarch
    python-gofer-qpid-1.3.0-1.el6sat.noarch
    python-isodate-0.5.0-1.pulp.el6sat.noarch
    python-kombu-3.0.15-12.pulp.el6sat.noarch
    python-pulp-bindings-2.4.0-0.30.beta.el6sat.noarch
    python-pulp-common-2.4.0-0.30.beta.el6sat.noarch
    python-pulp-puppet-common-2.4.0-0.30.beta.el6sat.noarch
    python-pulp-rpm-common-2.4.0-0.30.beta.el6sat.noarch
    python-qpid-0.22-14.el6sat.noarch
    python-qpid-qmf-0.22-33.el6.x86_64
    qpid-cpp-client-0.22-42.el6.x86_64
    qpid-cpp-server-0.22-42.el6.x86_64
    qpid-cpp-server-linearstore-0.22-42.el6.x86_64
    qpid-java-client-0.22-6.el6.noarch
    qpid-java-common-0.22-6.el6.noarch
    qpid-proton-c-0.7-1.el6.x86_64
    qpid-qmf-0.22-33.el6.x86_64
    qpid-tools-0.22-12.el6.noarch
    ruby193-rubygem-katello-1.5.0-78.el6sat.noarch
    rubygem-hammer_cli_katello-0.0.4-10.el6sat.noarch
Comment 5 Bryan Kearney 2014-09-11 12:20:54 UTC 
This was delivered with Satellite 6.0 which was released on 10 September 2014.
Note You need to log in before you can comment on or make changes to this bug.