Bug 650165 - Incremental dump creation in Satellite 5.4 is flawed
Summary: Incremental dump creation in Satellite 5.4 is flawed
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite 5
Classification: Red Hat
Component: Server
Version: 540
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: ---
Assignee: Michael Mráka
QA Contact: Šimon Lukašík
URL:
Whiteboard:
Depends On:
Blocks: sat54-errata
TreeView+ depends on / blocked
 
Reported: 2010-11-05 13:04 UTC by Milan Zázrivec
Modified: 2011-02-01 16:16 UTC (History)
4 users (show)

Fixed In Version: spacewalk-backend-1.2.13-25
Doc Type: Bug Fix
Doc Text:
Cause: Consequence: Fix: Result:
Clone Of:
Environment:
Last Closed: 2011-02-01 16:16:24 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2011:0190 0 normal SHIPPED_LIVE Red Hat Network Satellite bug fix update 2011-02-01 16:15:56 UTC

Description Milan Zázrivec 2010-11-05 13:04:02 UTC
Description of problem:
From satellite_tools/disk_dumper/dumper.py:

_query_get_package_ids_by_date_limits = rhnSQL.Statement("""
        select package_id
          from rhnPackage rp, rhnChannelPackage rcp
         where rcp.channel_id = :channel_id
         and rcp.package_id = rp.id
         and (rcp.modified >= TO_Date(:lower_limit, 'YYYYMMDDHH24MISS')
              or rp.last_modified >= TO_Date(:lower_limit, 'YYYYMMDDHH24MISS')
             )
         and (rcp.modified <= TO_Date(:upper_limit, 'YYYYMMDDHH24MISS')
              or rp.last_modified <= TO_Date(:upper_limit, 'YYYYMMDDHH24MISS')
             )
     """)

rhnPackage.last_modified values are (ordinarilly) being populated during
satellite-sync w/ RHN and represent the time the package was modified
on RHN side (i.e. it is a value we take from RHN and store it so as it is).

rhnChannelPackage.modified values are being filed during satellite-sync
and are set to sysdate of the sync (they represent the moment when the
particular package became part of that particular channel, which is
the moment of the sync).

For a particular package to become part of an incremental dump (defined by
:lower_limit and :upper_limit placeholders), either of rhnPackage.last_modified
rhnChannelPackage.modified has to fit into the interval specified.

Imagine you've just synced a RHEL-5-Server channel from RHN (i.e.
rhnChannelPackage.modified values are very fresh) and want to export its
content from recent past (for example starting week ago, finishing 2 days ago).
Your situation would look like:

lower_limit < upper_limit < all rhnChannelPackage.modified values

Since most of the rhnPackage.last_modified values will be older than
your specified lower_limit (the channel content is simply much older),

the

(rcp.modified >= TO_Date(:lower_limit, 'YYYYMMDDHH24MISS')
 or rp.last_modified >= TO_Date(:lower_limit, 'YYYYMMDDHH24MISS')
) and
(rcp.modified <= TO_Date(:upper_limit, 'YYYYMMDDHH24MISS')
 or rp.last_modified <= TO_Date(:upper_limit, 'YYYYMMDDHH24MISS')
)

condition will be true for most of the packages contained in the channel
and your incremental dump will contain pretty much the whole channel.

I believe rhn-satellite-exporter needs to be able to export the incremental
channel content based on rhnChannelPackage.modified values, and based on
rhnPackage.last_modified values but definitely must not 'OR' the two values.

In another words, when creating incremental content, you need to be able
to specify, whether you wish to select the packages based on the date
they were born in RHN or they were actually synced to your Satellite.


Version-Release number of selected component (if applicable):
spacewalk-backend-1.2.13-11

How reproducible:
Always

Steps to Reproduce:
1. New Satellite 5.4 installation, sync some content from RHN
2. Create incremental export starting 7 days ago, finishing 2 days ago
3. Check the created export
  
Actual results:
The export contains whole channel.

Expected results:
You need to be able to specify what date you want to select the packages
for the export.

Additional info:
https://bugzilla.redhat.com/show_bug.cgi?id=589524

Comment 2 Milan Zázrivec 2010-11-18 14:53:25 UTC
Needless to say, the same logic needs to apply to erratas, ks trees and
ks files (satellite_tools/disk_dumper/iss.py)

Comment 3 Michael Mráka 2010-11-22 10:04:08 UTC
We agreed to create new options --use-sync-date and --use-rhn-date and let user choose which one use (default is --use-rhn-date) rather than trying how to put them together.

Spacewalk git:
commit 44e62bc2296f5a36b9e736ef094d5533ea6f4f2a
    650165 - let user specify which date for incremental export use
commit 2e8568e823c0df4fc6cf6e8b4faff6a2a9183bca
    650165 - propagate use_rhn_date down to Dumper

Comment 4 Miroslav Suchý 2010-12-16 17:25:01 UTC
cherry-picked to satellite.git as commits:
19d94732b60dd63cc0801eba12696b1e2c992482
1965892a8e153fb11fdf7fccd39f1592869b80e3

Comment 6 Šimon Lukašík 2011-01-11 12:39:33 UTC
Flipping back to Assigned - for review

I have done very incremental export of channel, which should not contain
any packages. Directories packages_short and packages does not exists,
although channel.xml does contain all packages.

This breaks the import of the dump.

Comment 8 Michael Mráka 2011-01-17 13:22:01 UTC
Fixed in spacewalk git
commit ab481c31b699933cd186f9b68f5a0e4781a2b100
    650165 - _ChannelDumper should also understand use_rhn_date

Comment 9 Michael Mráka 2011-01-17 13:23:35 UTC
Backported to satellite git as
commit 88fa61bc4aa9b2a541a656ed73d9af1bc06963b1
    650165 - _ChannelDumper should also understand use_rhn_date
    (cherry picked from commit ab481c31b699933cd186f9b68f5a0e4781a2b100)

Comment 11 Šimon Lukašík 2011-01-24 17:17:31 UTC
Changing to Verified:

Testing procedure:
 - Verified scenario from comment 0
 - Created various dumps with using new options 
   (--use-rhn-date, --use-sync-date)
 - Manual check of dump/channels/*/channel.xml.gz for number of pkgs.
 - Verified that all the dumps created with new packages were importable
 - At least some of dumps were imported on different satellite

Example of dumps created, for both of the options (x represents of sync):
<x-2,x-2>, <x-7,x-2>, <x-7,x+2>, <x-365,x-2>, <x-2,x-2>, <0, x-2>

Verified against:
spacewalk-backend-tools-1.2.13-25.el5sat

Note:
Most of the testing was done on x86_64 only.

Comment 12 Florian Nadge 2011-01-27 11:26:04 UTC
Please be so kind and add a few key words to the technical note of this
bugzilla entry using the following structure:

Cause:

Consequence:

Fix:

Result:


For details, see:
https://bugzilla.redhat.com/page.cgi?id=fields.html#cf_release_notes

Thanks

Comment 13 Florian Nadge 2011-01-27 11:26:04 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Cause:

Consequence:

Fix:

Result:

Comment 14 Miroslav Suchý 2011-01-27 12:11:11 UTC
There is already nice text in errata.

Comment 15 errata-xmlrpc 2011-02-01 16:16:24 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2011-0190.html


Note You need to log in before you can comment on or make changes to this bug.