Bug 495816 - yum metadata generation in /var/cache/rhn can cause extreme server load
Summary: yum metadata generation in /var/cache/rhn can cause extreme server load
Keywords:
Status: CLOSED DEFERRED
Alias: None
Product: Red Hat Satellite 5
Classification: Red Hat
Component: Server
Version: 511
Hardware: All
OS: Linux
urgent
high
Target Milestone: ---
Assignee: Pradeep Kilambi
QA Contact: Red Hat Satellite QA List
URL:
Whiteboard:
Depends On: 495814
Blocks: 459674
TreeView+ depends on / blocked
 
Reported: 2009-04-14 22:34 UTC by Xixi
Modified: 2018-10-27 16:01 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of: 495814
Environment:
Last Closed: 2012-12-24 14:43:48 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Xixi 2009-04-14 22:34:17 UTC
Cloning for sat51maint

+++ This bug was initially created as a clone of Bug #495814 +++

If the cache for a given channel needs to get regenerated in /var/cache/rhn every client request for that new metadata kicks of a process to regenerate the files.

This can cause extreme load on the Satellite server and each thread is essentially doing the same thing.

In order to reproduce this issue I wrote a simple multi-threaded python utility to spawn multiple yum requests to a RHN Satellite server.  This client spins up 10 threads each doing:

yum clean all && yum search zsh

with separate --installroot parameters to allow simultaneous execution.

After setting up 2 RHEL5 clients each with my load simulator I was 
quickly able to get my Satellite to reach a load of *40-80* with it 
eventually ceasing to be accessible.

** Steps to reproduce the yum 'metadata storm' on a 5.2 Satellite:

1) Register at least 2 RHEL5 clients to your Satellite

2) Make sure your RHEL5 channel is populated and synced

3) Check out: 
http://svn.rhndev.redhat.com/viewcvs/trunk/eng/scripts/load-testing/yum-load-test.py

4) On each RHEL5 client as root execute: 'python yum-load-test.py'

5) On your RHN Satellite server run: 'rm -rf /var/cache/rhn/'

6) wait .. This will cause each client request to start re-generation of 
the metadata for the rhel5 channel.  As these requests pile up the 
server is quickly brought to its knees.

The more clients you have the quicker it will die.

Comment 2 Xixi 2009-04-14 22:43:04 UTC
bug 495814 for sat52maint
bug 495816 for sat51maint
bug 495815 for sat530-triage

Comment 14 Clifford Perry 2012-12-24 14:43:48 UTC
As per EOL Errata:
https://rhn.redhat.com/errata/RHSA-2012-1546.html

This is the End Of Life notification for RHN Satellite Server 5 versions
released to run on Red Hat Enterprise Linux 4.

On December 1st, 2012, per the life-cycle support policy, the following
versions of Satellite and Proxy products, released on Red Hat Enterprise
Linux 4, exited Production Phase 2 marking the end of their support by
Red Hat.

RHN Satellite & RHN Proxy:
- 5.0
- 5.1
- 5.2 on Red Hat Enterprise Linux 4
- 5.3 on Red Hat Enterprise Linux 4


I am closing out this specific bug as CLOSED, since it was tracking a product version, which is now EOL. 

Cliff


Note You need to log in before you can comment on or make changes to this bug.