Bug 518658 - delta crash with xz deltas
Summary: delta crash with xz deltas
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: createrepo
Version: rawhide
Hardware: All
OS: Linux
low
medium
Target Milestone: ---
Assignee: Luke Macken
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 518656 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2009-08-21 14:39 UTC by Bill Nottingham
Modified: 2016-09-20 02:40 UTC (History)
6 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2009-09-03 20:36:01 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
minor deltarpm compilation fix (547 bytes, patch)
2009-08-21 18:46 UTC, Bill Nottingham
no flags Details | Diff
really basic bindings (5.77 KB, patch)
2009-08-21 18:47 UTC, Bill Nottingham
no flags Details | Diff
createrepo: Use said bindings (2.44 KB, patch)
2009-08-21 18:48 UTC, Bill Nottingham
no flags Details | Diff

Description Bill Nottingham 2009-08-21 14:39:03 UTC
Description of problem:

When making rawhide:

Traceback (most recent call last):
  File "/usr/bin/mash", line 96, in <module>
    main()
  File "/usr/bin/mash", line 77, in main
    rc = themash.doCompose()
  File "/usr/lib/python2.6/site-packages/mash/__init__.py", line 378, in doCompose
    repocache = repocache, comps = True, arch = arch)
  File "/usr/lib/python2.6/site-packages/mash/__init__.py", line 215, in _write_files
    status = self._makeMetadata(repo_path, repocache, arch, comps, previous = previous_path)
  File "/usr/lib/python2.6/site-packages/mash/__init__.py", line 115, in _makeMetadata
    md.run(path)
  File "/usr/lib/python2.6/site-packages/mash/metadata.py", line 191, in run
    self.obj.run(path)
  File "/usr/lib/python2.6/site-packages/mash/metadata.py", line 155, in run
    self.repomatic.doPkgMetadata()
  File "/usr/lib/python2.6/site-packages/createrepo/__init__.py", line 367, in doPkgMetadata
    self.closeMetadataDocs()
  File "/usr/lib/python2.6/site-packages/createrepo/__init__.py", line 601, in closeMetadataDocs
    self.deltafile.write(self.generate_delta_xml())
  File "/usr/lib/python2.6/site-packages/createrepo/__init__.py", line 698, in generate_delta_xml
    drpm_rel_fn)
  File "/usr/lib/python2.6/site-packages/createrepo/deltarpms.py", line 48, in __init__
    self._getOldInfo(fo)
  File "/usr/lib/python2.6/site-packages/createrepo/deltarpms.py", line 71, in _getOldInfo
    if compobj.read(4)[:3] != "DLT":
  File "/usr/lib/python2.6/gzip.py", line 219, in read
    self._read(readsize)
  File "/usr/lib/python2.6/gzip.py", line 255, in _read
    self._read_gzip_header()
  File "/usr/lib/python2.6/gzip.py", line 156, in _read_gzip_header
    raise IOError, 'Not a gzipped file'
IOError: Not a gzipped file
mash failed in /mnt/koji/mash/rawhide-20090821/development

Version-Release number of selected component (if applicable):

xz-4.999.8-0.10.beta.20090817git.fc12.x86_64
deltarpm-3.5-0.git.20090729.fc12.1.x86_64
createrepo-0.9.7-15.fc12.noarch

How reproducible:

100%

Steps to Reproduce:
1. do xz -> xz deltaing

Comment 1 Bill Nottingham 2009-08-21 14:39:16 UTC
*** Bug 518656 has been marked as a duplicate of this bug. ***

Comment 2 Jonathan Dieter 2009-08-21 14:57:51 UTC
Do we have any python bindings for xz compression?  'Cause if not, this is going to suck.

Comment 3 Bill Nottingham 2009-08-21 15:00:17 UTC
Not that I can find.

Comment 4 Jonathan Dieter 2009-08-21 18:30:58 UTC
Ok, there's http://pypi.python.org/pypi/pyliblzma.  As far as I can see, it uses xz rather than lzma, but haven't tested it yet (and last release was in February).  I'll be away over the next few days, but if no one has done anything with this by the time I get back, I'll see what I can do.

Comment 5 Bill Nottingham 2009-08-21 18:46:54 UTC
Created attachment 358272 [details]
minor deltarpm compilation fix

Here come some patches; first fixes some needed includes in deltarpm header files.

Comment 6 Bill Nottingham 2009-08-21 18:47:45 UTC
Created attachment 358273 [details]
really basic bindings

Patch #2: Adds some basic deltarpm python bindings for reading delta rpms.

Comment 7 Bill Nottingham 2009-08-21 18:48:56 UTC
Created attachment 358274 [details]
createrepo: Use said bindings

And this patches createrepo to use these bindings.

Comments? (I have no idea if upstream deltarpm wants to take this.)

Comment 8 seth vidal 2009-08-24 18:24:02 UTC
I'm a little afraid of the basic deltarpm python bindings and the related patch to createrepo unless this change gets upstream. Otherwise it binds createrepo to a very special ver of deltarpm which I'd like to avoid.

Comment 9 Bill Nottingham 2009-08-24 18:36:41 UTC
I can see that; I just think it's simplest in the long run if createrepo doesn't have to parse the deltarpms itself.

Comment 10 seth vidal 2009-08-24 18:44:26 UTC
no dispute - but I don't want to put this patch into createrepo unless we've gotten it signed off by upstream deltarpm maintainers.

Comment 11 Jonathan Dieter 2009-08-24 18:51:52 UTC
I've sent an e-mail (cc'd to Bill) to Michael Schroeder (upstream deltarpm) asking if he has a problem with adding these bindings to deltarpm, and have still not received a response.  I have no problem adding them in, but, for something this large, I really want his ACK.

If he's fine with it, I'll add the patch upstream.

Comment 12 Jonathan Dieter 2009-08-31 14:36:50 UTC
Ok, the first two patches have been committed to upstream deltarpm and are available at http://koji.fedoraproject.org/koji/taskinfo?taskID=1645765

The bindings are now in a subpackage cunningly called python-deltarpm.

Comment 13 seth vidal 2009-09-03 20:36:01 UTC
applied the patch and it is koji as createrepo-0.9.8-2.fc12

If it works okay - I'll apply it upstream.


Note You need to log in before you can comment on or make changes to this bug.