Bug 620486 - rhn-satellite-exporter exports incomplete channels information for errata
Summary: rhn-satellite-exporter exports incomplete channels information for errata
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite 5
Classification: Red Hat
Component: Server
Version: 530
Hardware: All
OS: Linux
low
medium
Target Milestone: ---
Assignee: Michael Mráka
QA Contact: Dimitar Yordanov
URL:
Whiteboard:
Depends On:
Blocks: sat541-blockers
TreeView+ depends on / blocked
 
Reported: 2010-08-02 16:37 UTC by Stephan Dühr
Modified: 2018-11-14 14:43 UTC (History)
4 users (show)

Fixed In Version: spacewalk-backend-1.2.13-43
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-06-17 02:41:12 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
Script for parsing errata XML files generated by rhn-satellite-exporter (3.11 KB, text/x-python)
2010-08-05 16:45 UTC, Stephan Dühr
no flags Details

Description Stephan Dühr 2010-08-02 16:37:08 UTC
Description of problem:
the rhn-erratum-*.xml.gz files generated by rhn-satellite-exporter contain incomplete channels information when exporting channels cloned by API call channel.software.clone with original_state = False

Version-Release number of selected component (if applicable):
spacewalk-backend-tools-0.5.28-55.el5sat

How reproducible:
Always

Steps to Reproduce:
1. clone a base channel using API like this (python code):
client.channel.software.clone(session, 'rhel-x86_64-server-5,
    {'label': 'clone-test-rhel-x86_64-server-5',
     'name': 'clone-test-rhel-x86_64-server-5',
     'summary': 'clone-test-rhel-x86_64-server-5'},
    False)

2. export the channel using rhn-satellite-exporter
rhn-satellite-exporter --dir=/backup/dump/2010-07-30/rhn-sat-export/clone-test-rhel-x86_64-server-5 --channel=clone-test-rhel-x86_64-server-5

3. look at some of the rhn-erratum-*.xml.gz files
  
Actual results:
Example: errata/3/rhn-erratum-5283.xml.gz:
<?xml version="1.0" encoding="UTF-8"?><rhn-satellite generation="2" version="3.4"><rhn-errata><rhn-erratum advisory="RHSA-2010:0490-1" org_id="" channels="clone-test-rhel-i386-server-5 rhel-i386-as-4 rhel-i386-client-5 rhel-i386-client-workstation-5 rhel-i386-server-5 rhel-i386-ws-4 rhel-x86_64-as-4 rhel-x86_64-server-5" cve-names="CVE-2010-1748 CVE-2010-0540 CVE-2010-0542" packages="rhn-package-34483 rhn-package-34484 rhn-package-34485 rhn-package-34486 rhn-package-34487 rhn-package-34488 rhn-package-34489 rhn-package-34490 rhn-package-34505 rhn-package-34506 rhn-package-34507 rhn-package-34508 rhn-package-34516 rhn-package-34517" id="rhn-erratum-5283">

Expected results:
<?xml version="1.0" encoding="UTF-8"?><rhn-satellite generation="2" version="3.4"><rhn-errata><rhn-erratum advisory="RHSA-2010:0490-1" org_id="" channels="clone-test-rhel-i386-server-5 clone-test-rhel-x86_64-server-5 rhel-i386-as-4 rhel-i386-client-5 rhel-i386-client-workstation-5 rhel-i386-server-5 rhel-i386-ws-4 rhel-x86_64-as-4 rhel-x86_64-server-5" cve-names="CVE-2010-1748 CVE-2010-0540 CVE-2010-0542" packages="rhn-package-34483 rhn-package-34484 rhn-package-34485 rhn-package-34486 rhn-package-34487 rhn-package-34488 rhn-package-34489 rhn-package-34490 rhn-package-34505 rhn-package-34506 rhn-package-34507 rhn-package-34508 rhn-package-34516 rhn-package-34517" id="rhn-erratum-5283">

Additional info:
The API returns the complete list of channels:
>>> echannels = client.errata.applicableToChannels(session, 'RHSA-2010:0490')
>>> print ' '.join(map(lambda c: c['label'], echannels))
clone-test-rhel-i386-server-5 clone-test-rhel-x86_64-server-5 rhel-i386-as-4 rhel-x86_64-as-4 rhel-i386-client-5 rhel-i386-server-5 rhel-x86_64-server-5 rhel-i386-ws-4 rhel-i386-client-workstation-5

There are 3 exported errata that contain the correct channels list:
zgrep clone-test-rhel-x86_64-server-5 */*.xml.gz
0/rhn-erratum-3010.xml.gz:<?xml version="1.0" encoding="UTF-8"?><rhn-satellite generation="2" version="3.4"><rhn-errata><rhn-erratum advisory="RHBA-2009:1374-1" org_id="" channels="clone-test-rhel-x86_64-server-5 rhel-x86_64-server-5" cve-names="" packages="rhn-package-28437" id="rhn-erratum-3010"><rhn-erratum-advisory-name>RHBA-2009:1374</rhn-erratum-advisory-name><rhn-erratum-advisory-rel>1</rhn-erratum-advisory-rel><rhn-erratum-advisory-type>Bug Fix Advisory</rhn-erratum-advisory-type><rhn-erratum-product>Red Hat Enterprise Linux</rhn-erratum-product><rhn-erratum-description>mcelog is a utility that allows the root user to decode machine check
2/rhn-erratum-4242.xml.gz:<?xml version="1.0" encoding="UTF-8"?><rhn-satellite generation="2" version="3.4"><rhn-errata><rhn-erratum advisory="RHEA-2010:0136-1" org_id="" channels="clone-test-rhel-x86_64-server-5 rhel-x86_64-server-5 rhel-x86_64-server-cluster-5 rhel-x86_64-server-cluster-storage-5 rhel-x86_64-server-vt-5" cve-names="" packages="rhn-package-31941 rhn-package-31942" id="rhn-erratum-4242"><rhn-erratum-advisory-name>RHEA-2010:0136</rhn-erratum-advisory-name><rhn-erratum-advisory-rel>1</rhn-erratum-advisory-rel><rhn-erratum-advisory-type>Product Enhancement Advisory</rhn-erratum-advisory-type><rhn-erratum-product>Red Hat Enterprise Linux</rhn-erratum-product><rhn-erratum-description>The kmod-be2net-rhel5u4-2.101.377r-1.0 packages provide kernel modules for
8/rhn-erratum-4498.xml.gz:<?xml version="1.0" encoding="UTF-8"?><rhn-satellite generation="2" version="3.4"><rhn-errata><rhn-erratum advisory="RHBA-2010:0247-1" org_id="" channels="clone-test-rhel-x86_64-server-5 rhel-x86_64-server-5" cve-names="" packages="rhn-package-32721" id="rhn-erratum-4498"><rhn-erratum-advisory-name>RHBA-2010:0247</rhn-erratum-advisory-name><rhn-erratum-advisory-rel>1</rhn-erratum-advisory-rel><rhn-erratum-advisory-type>Bug Fix Advisory</rhn-erratum-advisory-type><rhn-erratum-product>Red Hat Enterprise Linux</rhn-erratum-product><rhn-erratum-description>mcelog is a daemon that collects and decodes Machine Check Exception data

Comment 1 Clifford Perry 2010-08-04 13:35:49 UTC
I wonder if this results from bug 591291 - Tomas?

Comment 2 Tomas Lestach 2010-08-04 15:42:58 UTC
This doesn't seem to be somehow connected with bug 591291. There're different API calls involved in every BZ and there doesn't seem to be a shared code.

If errata.applicableToChannels returns correct list of channels, it doesn't look like a problem in API.

However, I followed the reproducer, but didn't see the misbehavior.
1. I cloned the channel accrording to 1.)
2. I run a lighter export:
rhn-satellite-exporter --dir=clone-test-rhel-x86_64-server-5 --channel=clone-test-rhel-x86_64-server-5 --no-rpms --no-packages --no-kickstarts
3. erratum files look good and contain the correct channel list
in clone-test-rhel-x86_64-server-5/errata dir:

# zgrep RHSA-2010:0490-1 */*.xml.gz 
0/rhn-erratum-1520.xml.gz:<?xml version="1.0" encoding="UTF-8"?><rhn-satellite generation="2" version="3.5"><rhn-errata><rhn-erratum advisory="RHSA-2010:0490-1" org_id="" channels="clone-test-rhel-x86_64-server-5 rhel-x86_64-server-5" cve-names="CVE-2010-0542 CVE-2010-0540 CVE-2010-1748" packages="rhn-package-9524 rhn-package-9525 rhn-package-9526 rhn-package-9527 rhn-package-9528 rhn-package-9529" id="rhn-erratum-1520">

it looks like all the errata xml.gz files contain the correct channel list
# zgrep -l clone-test-rhel-x86_64-server-5 */*.xml.gz | wc -l
1532
# find -type f | wc -l
1532
# zgrep -l channels=\"clone-test-rhel-x86_64-server-5\ rhel-x86_64-server-5\" */*.xml.gz | wc -l
1532


Stephan, could you provide more detailed information, how to reproduce it?
Do you run the API as sat admin, or org admin?

Comment 3 Stephan Dühr 2010-08-05 16:43:23 UTC
hmm, this is strange. One difference is that I have more channels in my satellite.

I tried again with rhn-satellite-exporter --dir=/backup/dump/sat-exporter-test --channel=clone-test-rhel-x86_64-server-5 --no-rpms --no-packages --no-kickstarts

I'm analyzing the results with the attached script:

./errata-xml-parse-test.py /backup/dump/sat-exporter-test/errata clone-test-rhel-x86_64-server-5

results:
...
RHBA-2009:0220-2     4 False
RHBA-2009:1244-2     4 False
RHEA-2008:0370-7     5 False
Total Errata Files:  1552
Errata Files Parsed: 1552
Max. Channel Count:  8
Found Channel Label clone-test-rhel-x86_64-server-5: 6 times

./errata-xml-parse-test.py /backup/dump/sat-exporter-test/errata clone-test-rhel-x86_64-server-5 | fgrep True
RHSA-2010:0580-1     5 True
RHSA-2010:0585-1     4 True
RHBA-2010:0247-1     2 True
RHEA-2010:0136-1     5 True
RHBA-2009:1374-1     2 True
RHSA-2010:0578-1     8 True

Finally I found out that the problem is caused by rhn-satellite-exporter using outdated cached xml files in /var/cache/rhn/xml-errata/ via /usr/share/rhn/common/rhnCache.py

So if you now clone rhel-i386-server-5 to clone-test-rhel-i386-server-5 and run
rhn-satellite-exporter --dir=clone-test-rhel-i386-server-5
--channel=clone-test-rhel-i386-server-5 --no-rpms --no-packages
--no-kickstarts
you should see the same effect.

Shouldn't rhnCache.py invalidate outdated files once in a while?

Comment 4 Stephan Dühr 2010-08-05 16:45:35 UTC
Created attachment 436910 [details]
Script for parsing errata XML files generated by rhn-satellite-exporter

Comment 5 Tomas Lestach 2010-08-10 12:47:22 UTC
Since it looks like a backend issue, passing to Server component.

Comment 6 Carsten Clasohm 2010-08-24 13:42:33 UTC
Stephan, I'm currently at the customer where you encountered this problem. I have done a couple of tests using the method described in comment #2.

As you already found out, the root cause is invalid data in /var/cache/rhn/xml-errata. The easy fix is to remove the old /var/cache/rhn/xml-errata directory, which is automatically recreated when rhn-satellite-exporter is run.

I've done various tests with different lists of errata for the same channel, and it looks like it's only the old xml-errata directory which prevented updates. I think some old Satellite version caused this problem, and once we let the latest Satellite 5.3 create the directory from scratch, it's good.

Comment 7 Carsten Clasohm 2010-08-25 11:41:22 UTC
Correction for comment #6: The contents of /var/cache/rhn/xml-errata are not updated when we merge additional errata from the Red Hat into our clone channel. As a workaround, I'm now always deleting /var/cache/rhn/xml-errata before running rhn-satellite-exporter.

Comment 8 Aurelien Gouny 2011-02-24 02:51:00 UTC
Hitting this bug at a major Australian retailer with 4 Satellites.
Trying the workaround now, will let you guys know how it goes.

Aurelien (GPS Sydney, AU)

Comment 9 Aurelien Gouny 2011-02-24 21:51:43 UTC
Workaround seems to work fine to the customer.

Cheers,
Aurelien.

Comment 10 Michael Mráka 2011-04-18 10:04:24 UTC
Fixed in spacewalk master by
commit 2989f2487a80ba2aeca1953e280671200d36e6e5
    620486 - errata xml caching doesn't work for cloned channels

Fixed spacewalk package: spacewalk-backend-1.5.10-1

Comment 11 Michael Mráka 2011-04-18 11:09:24 UTC
Backported to SATELLITE-5.4 as
commit 7d7641c2ff7abe03031624f348de71a9d3c10d52
    620486 - errata xml caching doesn't work for cloned channels
   (cherry picked from commit 2989f2487a80ba2aeca1953e280671200d36e6e5)
   Conflicts:
        backend/satellite_tools/disk_dumper/dumper.py

Comment 13 Dimitar Yordanov 2011-05-17 13:01:30 UTC
Verified.

[root@ errata]#rhn-satellite-exporter --dir=clone-test-rhel-x86_64-server-5 --channel=clone-test-rhel-x86_64-server-5 --no-rpms --no-packages --no-kickstarts

[root@ errata]# find -type f | wc -l
1672

[root@ errata]# zgrep -l channels=\"clone-test-rhel-x86_64-server-5\.\*\ rhel-x86_64-server-5\.*\"  */*.xml.gz | wc -l
1672

Comment 14 Milan Zázrivec 2011-06-03 14:12:22 UTC
Verified in stage w/ spacewalk-backend-1.2.13-52 -> release pending.

Comment 15 Clifford Perry 2011-06-17 02:41:12 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

https://rhn.redhat.com/errata/RHEA-2011-0875.html


Note You need to log in before you can comment on or make changes to this bug.