Bug 970315

Description of problem:
Scientific Linux's yum repos contain some bz2 compressed xml files, but spacewalk-repo-sync does not currently have the code to handle this.

to be specific http://ftp.scientificlinux.org/linux/scientific/6/x86_64/os/repodata/updateinfo.xml.bz2 is one of the culprits.
as it stands most of their yum repos seem to contain updateinfo.xml file compressed in bz2 format.

it seams like yum already handles bz2 files but the yum plugins for spacewalk do not.

Version-Release number of selected component (if applicable):

In spacewalk 1.9 it causes this error

Repo http://ftp.scientificlinux.org/linux/scientific/6.4/x86_64/os/ has comps file comps-sl6-x86_64.xml.gz.
ERROR: Not a gzipped file
Sync completed.
Total time: 1:46:18

in Spacewalk 1.8 and lower it causes a trace in the taskomatic logs

it also causes the channels repodata to become inconsistent and in some cases non functional.

ng repos for channel: Scientific Linux 6 Updates FastBug (x86_64)
INFO   | jvm 1    | 2013/04/23 00:30:31 | 2013-04-23 00:30:31,913 [DefaultQuartzScheduler_Worker-9] ERROR com.redhat.rhn.manager.satellite.SystemCommandExecu
tor - Error encountered executing (args=[/usr/bin/spacewalk-repo-sync, --channel, scientific6-x86_64-updates-fast, --type, yum])
INFO   | jvm 1    | 2013/04/23 00:30:31 | 2013-04-23 00:30:31,913 [DefaultQuartzScheduler_Worker-9] ERROR com.redhat.rhn.manager.satellite.SystemCommandExecu
tor - Error message from process: Traceback (most recent call last):
INFO   | jvm 1    | 2013/04/23 00:30:31 |   File "/usr/bin/spacewalk-repo-sync", line 103, in <module>
INFO   | jvm 1    | 2013/04/23 00:30:31 |     sys.exit(abs(main() or 0))
INFO   | jvm 1    | 2013/04/23 00:30:31 |   File "/usr/bin/spacewalk-repo-sync", line 96, in main
INFO   | jvm 1    | 2013/04/23 00:30:31 |     sync.sync()
INFO   | jvm 1    | 2013/04/23 00:30:31 |   File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/reposync.py", line 112, in sync
INFO   | jvm 1    | 2013/04/23 00:30:31 |     self.error_msg("ERROR: %s" % e.value)
INFO   | jvm 1    | 2013/04/23 00:30:31 | AttributeError: 'exceptions.IOError' object has no attribute 'value'
INFO   | jvm 1    | 2013/04/23 00:30:31 |
INFO   | jvm 1    | 2013/04/23 00:30:31 | 2013-04-23 00:30:31,913 [DefaultQuartzScheduler_Worker-9] INFO  com.redhat.rhn.taskomatic.task.RepoSyncTask - Repo URL: http://ftp.scientificlinux.org/linux/scientific/6/x86_64/updates/fastbugs/
INFO   | jvm 1    | 2013/04/23 00:30:31 | Packages in repo:               106
INFO   | jvm 1    | 2013/04/23 00:30:31 | No new packages to sync.
INFO   | jvm 1    | 2013/04/23 00:30:31 |
INFO   | jvm 1    | 2013/04/23 00:30:31 | 2013-04-23 00:30:31,913 [DefaultQuartzScheduler_Worker-9] ERROR com.redhat.rhn.taskomatic.task.RepoSyncTask - Traceback (most recent call last):
INFO   | jvm 1    | 2013/04/23 00:30:31 |   File "/usr/bin/spacewalk-repo-sync", line 103, in <module>
INFO   | jvm 1    | 2013/04/23 00:30:31 |     sys.exit(abs(main() or 0))
INFO   | jvm 1    | 2013/04/23 00:30:31 |   File "/usr/bin/spacewalk-repo-sync", line 96, in main
INFO   | jvm 1    | 2013/04/23 00:30:31 |     sync.sync()
INFO   | jvm 1    | 2013/04/23 00:30:31 |   File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/reposync.py", line 112, in sync
INFO   | jvm 1    | 2013/04/23 00:30:31 |     self.error_msg("ERROR: %s" % e.value)
INFO   | jvm 1    | 2013/04/23 00:30:31 | AttributeError: 'exceptions.IOError' object has no attribute 'value'
INFO   | jvm 1    | 2013/04/23 00:30:31 |


How reproducible:

Consistent and easy

Steps to Reproduce:

create a new channel with http://ftp.scientificlinux.org/linux/scientific/6/x86_64/os/ as a repository sync source


Actual results:

files get synced and added to the database
none of the erratas get synced.
in spacewalk 1.8 reposync crashes before it completes resulting in bad and or incomplete file lists in the in yum which block updates and break kickstarts.

Expected results:

all files should get synced.
all erratas should get imported.
and the repodata should get generated properly.


Additional info:
places where the code need to be updated that I've already found are

around line 178 in /usr/lib/python2.6/site-packages/spacewalk/satellite_tools/reposync.py

and around 64 of /usr/lib/python2.6/site-packages/spacewalk/satellite_tools/repo_plugins/yum_src.py in class YumUpdateMetadata