Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1622972

Summary: cdn-sync do not update errata's update_date when it is updated on CDN
Product: Red Hat Satellite 5 Reporter: Jan Hutař <jhutar>
Component: Satellite SynchronizationAssignee: Michael Mráka <mmraka>
Status: CLOSED ERRATA QA Contact: Radovan Drazny <rdrazny>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 580CC: adujicek, mmraka, rdrazny, tlestach
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: spacewalk-backend-2.5.3-172 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-11-30 18:05:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jan Hutař 2018-08-28 09:44:30 UTC
Description of problem:
cdn-sync do not update errata's update_date when it is updated on CDN


Version-Release number of selected component (if applicable):
Satellite 5.8.0
spacewalk-backend-cdn-2.5.3-169.el6sat.noarch


How reproducible:
always


Steps to Reproduce:
1. Sync rhel-x86_64-server-7
2. Wait till some errata "update_date gets updated
3. Check that errata update_date in DB:

rhnschema=# select advisory, issue_date, update_date, last_modified from rhnErrata where advisory LIKE 'RHBA-2015:0738%';
     advisory     |       issue_date       |      update_date       |         last_modified         
------------------+------------------------+------------------------+-------------------------------
 RHBA-2015:0738-1 | 2015-03-26 00:00:00-04 | 2015-03-26 00:00:00-04 | 2017-08-21 13:00:01.655202-04
(1 row)

4. Resync the channel
5. Check if the update_date for the errata got updated
   (also tried after `cdn-sync --clear-cache; rm -rf /var/cache/rhn/reposync/* /var/cache/rhn/satsync/* /var/satellite/redhat/NULL/stage/* /var/cache/yum/*`)

rhnschema=# select advisory, issue_date, update_date, last_modified from rhnErrata where advisory LIKE 'RHBA-2015:0738%';
     advisory     |       issue_date       |      update_date       |         last_modified         
------------------+------------------------+------------------------+-------------------------------
 RHBA-2015:0738-1 | 2015-03-26 00:00:00-04 | 2015-03-26 00:00:00-04 | 2017-08-21 13:00:01.655202-04
(1 row)


6. Optionally compare with same errata synced from Sat6 content ISO:

rhnschema=# select advisory, issue_date, update_date, last_modified from rhnErrata where advisory LIKE 'RHBA-2015:0738%';
     advisory     |       issue_date       |      update_date       |         last_modified         
------------------+------------------------+------------------------+-------------------------------
 RHBA-2015:0738-9 | 2015-03-26 09:33:34-04 | 2018-07-10 13:53:38-04 | 2018-08-24 06:55:47.980812-04
(1 row)


Actual results:
"update_date" in step "3." and "5." is same, although CDN repodata says:

https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/os/repodata/ff9f14b45618c3e60c5ad9c99caaccd973224fe12267f3b4a4403c5bb16cb0a7-updateinfo.xml.gz

        <update status="final" from="release-engineering" version="9" type="bugfix" >
            <id>RHBA-2015:0738</id>
            <issued date="2015-03-26 09:33:34 UTC" />
            [...]
            <updated date="2018-07-10 13:53:38 UTC" />


Expected results:
All properties of errata should be updated when these are updated on the CDN


Additional info:
This feels strange as this way I got to the situation with Sat6 content ISO content being partly newer then content synced directly from CDN.

I have noticed the issue when comparing content on a Sat synced (this is for rhel-x86_64-server-7) from Sat6 content ISO and Sat synced from CDN a ~month ago and resynced now:

    RHBA-2015:0738:
      update_date: '2018-07-10 13:53:38' vs. '2015-03-26 00:00:00'
    RHSA-2015:0726:
      update_date: '2018-07-10 13:54:09' vs. '2015-03-26 00:00:00'
    RHBA-2015:0734:
      update_date: '2018-07-10 13:53:20' vs. '2015-03-26 00:00:00'
    RHEA-2015:0731:
      update_date: '2018-07-10 13:52:51' vs. '2015-03-26 00:00:00'
    RHBA-2015:0733:
      update_date: '2018-07-10 13:53:07' vs. '2015-03-26 00:00:00'
    RHSA-2015:0696:
      update_date: '2018-07-10 13:51:36' vs. '2015-03-17 00:00:00'
    RHSA-2015:0718:
      update_date: '2018-07-10 13:51:55' vs. '2015-03-24 00:00:00'
    RHSA-2015:0716:
      update_date: '2018-07-10 13:52:15' vs. '2015-03-23 00:00:00'
    RHBA-2015:0626:
      update_date: '2018-07-10 13:50:55' vs. '2015-03-05 00:00:00'
    RHSA-2015:0672:
      update_date: '2018-07-10 13:51:14' vs. '2015-03-10 00:00:00'
    RHBA-2017:2283:
      update_date: '2017-08-28 00:00:00' vs. '2017-08-01 05:36:41'
      issue_date: '2017-08-28 00:00:00' vs. '2017-08-01 05:36:41'
    RHBA-2015:0747:
      update_date: '2018-07-10 13:52:37' vs. '2015-03-26 00:00:00'
    RHBA-2015:0739:
      update_date: '2018-07-10 13:53:29' vs. '2015-03-26 00:00:00'

Comment 1 Jan Dobes 2018-08-28 15:06:43 UTC
I think it's caused by some optimizations in cdn-sync and certain attributes are not updated by default in the DB when changed. Also there is an --force-all-errata attribute that should fix all dates.

Comment 2 Michael Mráka 2018-10-02 10:03:18 UTC
Actually this looks more like an issue with creating Sat 6 content ISOs.

If I download RHEL7 server updateinfo.xml from CDN today I can see
e.g. RHBA-2015:0738, RHSA-2015:0726 and RHBA-2015:0734 with '2015-03-26 00:00:00' timestamp (both in updateinfo and satellite db).



Anyway I've changed the code to update erratum if update date has changed.

Fix in spacewalk git 
commit 57cb53f70e2c9aaa5ffab06e4270c9068c854050
    1622972 - sync errata if update date has changed

Comment 4 Michael Mráka 2018-10-05 13:20:20 UTC
Satellite fix

commit 383b0c9284028943fef44718cd5de483a25f08ff
    1622972 - sync errata if update date has changed

Comment 5 Radovan Drazny 2018-10-10 10:22:35 UTC
Syncing using the cdn-sync from the CDN works, and the "update_date" and "last_modified" fields are properly updated. 

There is a problem when using satellite-sync to import channel from content ISOs. I'm not quite sure if this is really connected to the current BZ. Only thing that points this way is the mention of 'last_modified' in the error messsage:

# satellite-sync -m .  -c rhel-x86_64-client-7
06:07:44 Red Hat Satellite - file-system synchronization
06:07:44    mp:  /root/dumps/extracted
06:07:44    db:  rhnuser/<password>@rhnschema
06:07:44 
06:07:44 Retrieving / parsing channel-families data
06:07:44 channel-families data complete
06:07:44 
06:07:44 Retrieving / parsing product names data
06:07:45 
06:07:45 Retrieving / parsing arches data
06:07:45 arches data complete
06:07:45 
06:07:45 Retrieving / parsing additional arches data
06:07:45 additional arches data complete
06:07:45 
06:07:45 Retrieving / parsing channel data
06:07:47    p = previously imported/synced channel
06:07:47    . = channel not yet imported/synced
06:07:47    base-channels:
06:07:47       p rhel-x86_64-client-7                     16179       full import from Fri Apr 20 11:52:11 2018
06:07:47 
06:07:47 Channel data complete
06:07:47 
06:07:47 Retrieving short package metadata (used for indexing)
06:07:47    Retrieving / parsing short package metadata: rhel-x86_64-client-7 (16179)
06:07:56 Diffing package metadata (what's missing locally?): rhel-x86_64-client-7
            ________________________________________
Diffing:    ######################################## - complete
06:08:18 
06:08:18 Downloading package metadata
+++ sending log as an email +++

SYNC ERROR: unhandled exception occurred:

(Check logs/email for potentially more detail)

TypeError("'NoneType' object is unsubscriptable",)
'NoneType' object is unsubscriptable

Exception reported from ibm-x3250m4-02.rhts.eng.bos.redhat.com
Time: Wed Oct 10 06:08:33 2018
Exception type <type 'exceptions.TypeError'>

Exception Handler Information
Traceback (most recent call last):
  File "/usr/bin/satellite-sync", line 138, in main
    return satsync.Runner().main()
  File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/satsync.py", line 245, in main
    ret = method()
  File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/satsync.py", line 351, in _step_download_packages
    return self.syncer.download_package_metadata()
  File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/satsync.py", line 1163, in download_package_metadata
    missing_packages = self._missing_not_cached_packages()
  File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/satsync.py", line 1153, in _missing_not_cached_packages
    or package_collection.get_package(pid)['last_modified']
TypeError: 'NoneType' object is unsubscriptable

Michal, should this error be considered a part of the current bug, or is it a new standalone bug?

Comment 6 Radovan Drazny 2018-10-15 12:23:27 UTC
Ok, I am unable to re-reproduce the issue in the c#5 now. After an off-line discussion with Michal we have decided to leave this as is, as the problem appeared in satellite-sync and not in the cdn-sync (which is the tool fixed in this BZ). Other than this, the last_modified date gets updated correctly on a CDN sync.

Comment 8 Michael Mráka 2018-10-19 13:46:00 UTC
Fixed oracle failure in spacewalk git by
commit ca0553392f78a7842f167f79420e60497eb94d91
    1622972 - convert datetime in python
    otherwise postgresql and oracle need different selects/casts

Comment 9 Michael Mráka 2018-10-19 14:18:37 UTC
backported to SATELLITE-5.8 as
commit ee12f9357b52c023976e0c628d7563157d416fd5
    1622972 - convert datetime in python

Comment 10 Radovan Drazny 2018-10-25 07:43:31 UTC
Re-verified on spacewalk-backend-2.5.3-172. Repo sync works on Oracle now.

Comment 12 errata-xmlrpc 2018-11-30 18:05:12 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3756