Bug 729657

Summary: rhn_check fills /var by attempting to do a full download of the metadata
Product: Red Hat Enterprise Linux 5 Reporter: Felix Dewaleyne <fdewaley>
Component: yum-rhn-pluginAssignee: Michael Mráka <mmraka>
Status: CLOSED ERRATA QA Contact: Pavel Studeník <pstudeni>
Severity: high Docs Contact:
Priority: urgent    
Version: 5.7CC: cperry, cww, ekuric, fdewaley, goetz.dirk, james.antill, jentrena, jhutar, luvilla, mishu, mmello, mzazrivec, ndevos, pep, pgervase, pstudeni, ubeck, voetelink, xdmoon
Target Milestone: rcKeywords: Regression, ZStream
Target Release: 5.8   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: yum-rhn-plugin-0.5.4-24.el5 Doc Type: Bug Fix
Doc Text:
The rhnplugin provided by the yum-rhn-plugin package downloaded all the repository metadata available, including others and filelist which were not required by the regular usage of yum. With this updated package, rhnplugin now downloads only relevant subpart of metadata.
Story Points: ---
Clone Of:
: 732355 (view as bug list) Environment:
Last Closed: 2012-02-21 06:28:52 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 732355, 732427    

Description Felix Dewaleyne 2011-08-10 13:34:45 UTC
Description of problem:
rhn-check of 5.7 downloads all the repodata available, including others and filelist which are not required by the regular usage of yum (aka no yum provide or yum-security usage)

Version-Release number of selected component (if applicable):
rhn-check.noarch                    0.4.20-56.el5


How reproducible:
all the time


Steps to Reproduce:
1.make sure that the version is installed
2.install osad on the machine and start it
3.yum clean all on the client
4.plan any package installation for that machine

  
Actual results:
du -sh /var/cache/yum will be large and the cache will have others and filelist info. for a machine registered to a 32 bit base channel and a rhn-channel we have 635+mb consumed in the cache because of the filelist and others.

Expected results:
the space consumed should be about 35mb large, filelist and others should not be downloaded.


Additional info:
the expected behaviour could be observed with the previous version of rhn-check, 0.4.20-46.el5.

Comment 6 Milan Zázrivec 2011-08-12 08:41:21 UTC
There are few things that need to be explained wrt to this report.

1. It's not true, that for regular yum / rhn_check usage (i.e. installing
a package) all you need is primary.xml. There are situations in which
yum (or yum libraries) will download filelist.xml -- in cases it needs
to resolve file dependencies (try to install mod_ssl or httpd for example).

others.xml -- on the other hand -- is not necessary for this sort of usage
(yes, others.xml is downloaded during rhn_check although it does not need
to be).

2. At some point in the past, yum started to create .sqlite files for each
of the metadata files it downloads; these .sqlite files tend to be quite
large (depending on the size of the corresponding .xml)

3. For RHEL-5.7 and for bug #627525 we had to restructure yum-rhn-plugin
a bit, part of this revamp was the following commit:

http://git.fedorahosted.org/git/?p=spacewalk.git;a=commitdiff;h=103100a26ed7fb6e6aba71a76c07b964bd0f7a06

which causes rhn_check to download all available metadata from given repo.


Now my comments on the three points above:

1) Nothing we can do. Yum will download filelist.xml (and create .sqllite
for it) for more complicated dependency resolution whether you like it or not.

2) Nothing we can do either. Yum design decision.

3) We will have to return yum-rhn-plugin to the state it was before RHEL-5.7,
since the new design brought more troubles than advantages (this bug being one
of them).

Once the old structure of yum-rhn-plugin is restored, this bug will be
automatically resolved: that means others.xml will not be downloaded by
rhn_check.

I'm gonna ack this bug, move it to ASSIGNED and leave it open for a proper
QA verification. Otherwise, there's not much to do here.

Comment 9 Uwe Beck 2011-08-14 18:42:50 UTC
yum works other than rhn_check!

This problem was reportet for rhn_check and is relevant since RHEL6.1
and RHEL5.7.

rhn_check runs over rhnsd every four hours.
rhn_check download all metadata new if there was a change at RHN channels
at this time.
rhn_check do this also if there is no job at the system queue and also at
rhnreg_ks during %post at kickstart.

This problem was never seen before RHEL6.1 and RHEL5.7.

Please see the traffic for metadata from content-xmlrpc.rhn.redhat.com for
75 x RHEL5.7 and 45 x RHEL6.1 form 08.08.2011 to 14.08.2011:

08/Aug/2011: 6611098398 byte
09/Aug/2011: 7119152309 byte
10/Aug/2011: 2826012962 byte
11/Aug/2011: 12661492264 byte
12/Aug/2011: 8510889604 byte
13/Aug/2011: 0 byte
14/Aug/2011: 0 byte

This means 37728645537 byte absurd traffic for check the RHN queue with
no job to do for this 120 systems like I use RHN at this time, at this
week.

Who knows the normal bandwidth for a ADSL or SDSL data link with German
Telekom as provider?

Comment 15 Gary Smith 2011-08-18 09:06:21 UTC
*** Bug 728876 has been marked as a duplicate of this bug. ***

Comment 28 Michael Mráka 2011-08-23 08:52:49 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
The rhnplugin provided by the yum-rhn-plugin package downloaded all the repository metadata available, including others and filelist which were not required by the regular usage of yum. With this updated package, rhnplugin now downloads only relevant subpart of metadata.

Comment 29 Dennixx 2011-08-24 13:41:42 UTC
The yum-rhn-plugin-0.5.4-22.el5_7.1.noarch update seems to solve the problem!
Looks like I can enable rhnsd on my systems again....

Comment 30 Uwe Beck 2011-08-24 14:18:46 UTC
I do not understand why rhn_check must download the metadata
if there is no job at the queue for this system at RHN to do.

rhn_check do this on every change of metadata at the RHN
channels which a system is subscribed. 

The only good news at this time for me is:
rhn_check only downloads the primary.xml.gz files now. This means
we have now only 10% traffic with "content-xmlrpc.rhn.redhat.com"
than before.

Comment 37 errata-xmlrpc 2012-02-21 06:28:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2012-0162.html