Bug 783222

Summary: [abrt] jpackage-utils-1.7.5-15.fc16: zipfile.py:374:_decodeExtra:error: unpack requires a string argument of length 24
Product: [Fedora] Fedora Reporter: Daniel <bugzilla-daniel>
Component: jpackage-utilsAssignee: Alexander Kurtakov <akurtako>
Status: CLOSED WONTFIX QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 16CC: akurtako, dbhole, mael.lavault, patrickm
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Unspecified   
Whiteboard: abrt_hash:6f7e0399d2b66119efc45f8350ece52a7ab07599
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-02-13 08:26:55 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
ant build script that caused the problem
none
spec file used by rpmbuild when invoked from ant build
none
rpmbuild env that triggers bug none

Description Daniel 2012-01-19 17:49:04 UTC
libreport version: 2.0.8
abrt_version:   2.0.4.981
cmdline:        /usr/bin/python /usr/lib/rpm/osgi.prov
executable:     /usr/lib/rpm/osgi.prov
kernel:         3.1.1-2.fc16.x86_64
reason:         zipfile.py:374:_decodeExtra:error: unpack requires a string argument of length 24
time:           Wed 30 Nov 2011 12:09:34 PM MST
uid:            1000
username:       oathizhi

backtrace:
:zipfile.py:374:_decodeExtra:error: unpack requires a string argument of length 24
:
:Traceback (most recent call last):
:  File "/usr/lib/rpm/osgi.prov", line 94, in <module>
:    builder = TagBuilder ()
:  File "/usr/lib/rpm/osgi.prov", line 60, in __init__
:    self.get_osgi_provide (path)
:  File "/usr/lib/rpm/osgi.prov", line 87, in get_osgi_provide
:    jarfile = ZipFile(path)
:  File "/usr/lib64/python2.7/zipfile.py", line 712, in __init__
:    self._GetContents()
:  File "/usr/lib64/python2.7/zipfile.py", line 746, in _GetContents
:    self._RealGetContents()
:  File "/usr/lib64/python2.7/zipfile.py", line 805, in _RealGetContents
:    x._decodeExtra()
:  File "/usr/lib64/python2.7/zipfile.py", line 374, in _decodeExtra
:    counts = unpack('<QQQ', extra[4:28])
:error: unpack requires a string argument of length 24
:
:Local variables in innermost frame:
:ln: 43230
:self: <zipfile.ZipInfo object at 0x7f0769ddf9b0>
:unpack: <built-in function unpack>
:tp: 1
:extra: '\x01\x00\xde\xa8\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd\x01'

comment:
:doing an ant build
:
:[rpm] Building the RPM based on the feeder.spec file
:      [rpm] Processing files: perfectsearch-feeder-5.1.19.1378-0.noarch
:      [rpm] Traceback (most recent call last):
:      [rpm]   File "/usr/lib/rpm/osgi.prov", line 94, in <module>
:      [rpm]     builder = TagBuilder ()
:      [rpm]   File "/usr/lib/rpm/osgi.prov", line 60, in __init__
:      [rpm]     self.get_osgi_provide (path)
:      [rpm]   File "/usr/lib/rpm/osgi.prov", line 87, in get_osgi_provide
:      [rpm]     jarfile = ZipFile(path)
:      [rpm]   File "/usr/lib64/python2.7/zipfile.py", line 712, in __init__
:      [rpm]     self._GetContents()
:      [rpm]   File "/usr/lib64/python2.7/zipfile.py", line 746, in _GetContents
:      [rpm]     self._RealGetContents()
:      [rpm]   File "/usr/lib64/python2.7/zipfile.py", line 805, in _RealGetContents
:      [rpm]     x._decodeExtra()
:      [rpm]   File "/usr/lib64/python2.7/zipfile.py", line 374, in _decodeExtra
:      [rpm]     counts = unpack('<QQQ', extra[4:28])
:      [rpm] struct.error: unpack requires a string argument of length 24

Comment 1 Alexander Kurtakov 2012-01-19 19:04:41 UTC
How did you get this error?
Can you please attach the file that caused the problem?

Comment 2 Daniel 2012-01-19 19:21:14 UTC
I was doing an ant build. The build calls the ant "rpm" action. For a reason I do not understand, rpmbuild is trying to decompress something as it creates the RPM file--perhaps it needs to read the manifest from some system package/egg that it's trying to include in the final RPM? At any rate, whatever zipped data it's trying to read is causing a decompression error in python's zip library.

The error is very consistent, and it is associated with F16. Building the same code on the same hardware, when I was running F14, gave no errors. What's changed is python version (2.7 instead of 2.6).

I don't believe the problem is specific to a particular project; all ant builds that I do on my machine experience the same error when rpmbuild gets invoked through ant's "rpm" action. However, I do not see the error when I manually build RPMs on my machine.

I will attach the .spec file and the build.xml that I am using; please let me know if further attachments would be helpful.

Comment 3 Daniel 2012-01-19 19:22:10 UTC
Created attachment 556374 [details]
ant build script that caused the problem

Comment 4 Daniel 2012-01-19 19:22:42 UTC
Created attachment 556375 [details]
spec file used by rpmbuild when invoked from ant build

Comment 5 Alexander Kurtakov 2012-01-19 19:42:49 UTC
Can you please upload a srpm I can use for testing purposes?
I don't see any SourceX tag in this spec.

Comment 6 Daniel 2012-01-20 00:33:35 UTC
I have narrowed down this bug considerably and should be able to post something. However, I can't finish until tomorrow.

Comment 7 Daniel 2012-01-23 23:17:26 UTC
Created attachment 557098 [details]
rpmbuild env that triggers bug

simple environment that reproduces bug; extract to /tmp and follow instructions in comment

Comment 8 Daniel 2012-01-23 23:28:53 UTC
I have uploaded a very simple env that should reproduce the bug for you. To use it, extract archive to /tmp (creating /tmp/bug783222). Open terminal, go to that folder, and run the following cmd:

/usr/bin/rpmbuild --define "_topdir /tmp/feeder.tmp/rpm" -bb --buildroot /tmp/feeder.tmp/root --define "psname feeder" --define "psprefix perfectsearch" --define "psversion 5.1.19.1378" --define "psrelease 0" --define "psbuildroot /tmp/feeder.tmp/root" --define "psapplianceversion appliance3" SPECS/feeder.spec

It looks to me like the bug is caused by a layer of code trying to crack a .jar so its contents can be embedded in an RPM. When the .jar gets cracked, one .class file is somehow processed by the compression library, and that .class violates the compression library's expectations in a way I do not understand. The malformed .class file is located (after extracting the archive) inside /tmp/bug783222/root/opt/perfectsearch/appliance3/lib/tika-app-1.0.jar, at com/pixelmed/apps/DoseUtility$OurSourceDatabaseTreeBrowser$1.class. The class file comes from a compile of unmodified apache tika source code, against openjdk 6, on the same Fedora 16 machine that's running the rpmbuild command.

The way I determined that this particular .class file is at fault is that I progressively deleted more and more of the .class files in tika-app-1.0.jar until only the malformed .class remained. If you add any normal .class file back into the jar (e.g., just open the jar in archive mgr and drag a .class into it), and then subtract the malformed .class, the error will go away.

Comment 9 Andrew Overholt 2012-01-24 14:38:07 UTC
As a casual observer of this bug, I just wanted to thank you, Daniel, for such excellent tracking down and creating a test case!  Nice work.

Comment 10 Maël Lavault 2012-02-22 16:15:24 UTC
Hi,

I encounter the same bug when building webos sdk on fedora 16. Any chance to get this fixed soon ? 

Is there any workaround ?

Thanks !

Comment 11 Deepak Bhole 2012-02-22 16:32:04 UTC
*** Bug 796289 has been marked as a duplicate of this bug. ***

Comment 12 Alexander Kurtakov 2012-02-22 16:36:12 UTC
Is this really breaking the build or just printing the error on the console but
the build finishes? This sounds like a bug in python zip module not being able
to open given zips or some tool producing invalid zips.

Comment 13 Maël Lavault 2012-02-22 16:41:51 UTC
Yep it break the build. No rpm or rpmsrc. It's very annoying.

Here is what i get :


rpmbuild --target i386 --short-circuit -bi palm-sdk.spec
Construction pour plate-formes cibles: i386
Construction pour cible i386
Processing files: palm-sdk-3.0.5-1.svn528736.i386
Traceback (most recent call last):
  File "/usr/lib/rpm/osgi.prov", line 94, in <module>
    builder = TagBuilder ()
  File "/usr/lib/rpm/osgi.prov", line 60, in __init__
    self.get_osgi_provide (path)
  File "/usr/lib/rpm/osgi.prov", line 87, in get_osgi_provide
    jarfile = ZipFile(path)
  File "/usr/lib64/python2.7/zipfile.py", line 712, in __init__
    self._GetContents()
  File "/usr/lib64/python2.7/zipfile.py", line 746, in _GetContents
    self._RealGetContents()
  File "/usr/lib64/python2.7/zipfile.py", line 805, in _RealGetContents
    x._decodeExtra()
  File "/usr/lib64/python2.7/zipfile.py", line 371, in _decodeExtra
    tp, ln = unpack('<HH', extra[:4])
struct.error: unpack requires a string argument of length 4
Provides: osgi(javax.servlet) = 2.5.0 osgi(org.eclipse.jetty.continuation) = 7.1.6 osgi(org.eclipse.jetty.http) = 7.1.6 osgi(org.eclipse.jetty.io) = 7.1.6 osgi(org.eclipse.jetty.server) = 7.1.6 osgi(org.eclipse.jetty.util) = 7.1.6
Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1
Requires: /bin/bash /bin/sh
Vérification des fichiers non empaquetés: /usr/lib/rpm/check-files /home/moimael/rpmbuild/BUILDROOT/palm-sdk-3.0.5-1.svn528736.i386
erreur: Fichier(s) installé(s) (mais non empaquetés):
   /opt/PalmPDK/arm-gcc/arm-none-linux-gnueabi/bin/ar
   /opt/PalmPDK/arm-gcc/arm-none-linux-gnueabi/bin/as
   /opt/PalmPDK/arm-gcc/arm-none-linux-gnueabi/bin/c++
   /opt/PalmPDK/arm-gcc/arm-none-linux-gnueabi/bin/g++
   /opt/PalmPDK/arm-gcc/arm-none-linux-gnueabi/bin/gcc
   /opt/PalmPDK/arm-gcc/arm-none-linux-gnueabi/bin/ld
   /opt/PalmPDK/arm-gcc/arm-none-linux-gnueabi/bin/nm
   /opt/PalmPDK/arm-gcc/arm-none-linux-gnueabi/bin/objcopy
   /opt/PalmPDK/arm-gcc/arm-none-linux-gnueabi/bin/objdump
   /opt/PalmPDK/arm-gcc/arm-none-linux-gnueabi/bin/ranlib
   /opt/PalmPDK/arm-gcc/arm-none-linux-gnueabi/bin/strip
   /opt/PalmPDK/arm-gcc/arm-none-linux-gnueabi/include/c++/4.3.3/algorithm
   /opt/PalmPDK/arm-gcc/arm-none-linux-gnueabi/include/c++/4.3.3/arm-none-linux-gnueabi/bits/atomic_word.h

..... And a lot more file which seems to be extracted but the rpm is not build.

Comment 14 Alexander Kurtakov 2012-02-22 18:19:24 UTC
Actually to make testing easier can someone provide a simple/small srpm that can be used for reproducing the problem?

Comment 15 Daniel 2012-02-22 18:31:38 UTC
Alexander:

I cannot dup this with an SRPM. Maybe I'm just doing things wrong. However, please read comment #8. I attached a tiny archive that should dup the bug just as easily.

Comment 16 Maël Lavault 2012-02-22 18:35:17 UTC
srpm do not build too, so a spec file is better, like daniel said.

Comment 17 Fedora End Of Life 2013-01-16 10:08:28 UTC
This message is a reminder that Fedora 16 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 16. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '16'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 16's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 16 is end of life. If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora, you are encouraged to click on 
"Clone This Bug" and open it against that version of Fedora.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 18 Fedora End Of Life 2013-02-13 08:26:57 UTC
Fedora 16 changed to end-of-life (EOL) status on 2013-02-12. Fedora 16 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.