Bug 833427

Changing component to cpio.

$ rpm -q cpio
cpio-2.10-10.el6.x86_64

$ rpm2cpio matlab-2010b-3.el6.src.rpm | cpio -t
matlab-2010b.tar.gz
cpio: premature end of file

In fact creating a SRC RPM on RHEL 6.2, and then subsequently trying to install that src rpm breaks in the same way. This worked on RHEL 6.0. Of note here is that the SRC RPM is 4.8 GB big.

This is a pretty bad regression.

Thanks for your report. Afaik rpm uses its own implementation of cpio, so it is possible that it was broken for large files at the time of the RHEL-6.0 - and unpacking it directly via cpio just fails - as the file is probably incomplete or incompatible with original cpio format. Is there any reasoning why you moved this bugzilla from rpm to cpio? Just based on cpio complains about premature end of file? Have you tried using RHEL-6.0 cpio for unpacking?

I've a hard time seeing what change in rpm might have caused such a thing, but cpio bug this is not as it's never involved in creation or extraction of packages -> switching back to rpm component.

Since a previously working src.rpm fails to install (well, extract) on rhel-6.2 it'd seem that the payload gets truncated on extract phase (as opposed to creation), for whatever reason. Can you easily test whether the src.rpm created on rhel-6.2 is installable on 6.0?

(In reply to comment #6)
> I've a hard time seeing what change in rpm might have caused such a thing,
> but cpio bug this is not as it's never involved in creation or extraction of
> packages -> switching back to rpm component.
> 

OK, sorry, I naively assumed that rpm used cpio to create and extract rpms. Nonetheless, there is something that has changed to break *both* cpio and rpm.

> Since a previously working src.rpm fails to install (well, extract) on
> rhel-6.2 it'd seem that the payload gets truncated on extract phase (as
> opposed to creation), for whatever reason. Can you easily test whether the
> src.rpm created on rhel-6.2 is installable on 6.0?

I can't check that as I no longer have any 6.0 machines alas. 

But, what I can tell you is:

1) The RPM created under RHEL 6.0 cannot be installed by rpm from RHEL 6.2

2) The RPM created under RHEL 6.0 cannot be unpacked using cpio from RHEL 6.2

3) An RPM created using rpm from RHEL 6.2 cannot be installed using the same rpm binary from RHEL 6.2

4) An RPM created using rpm from RHEL 6.2 cannot be unpacked using cpio from RHEL 6.2

This is for an rpm containing a large (4.8 GB) tar.gz file.

So, I don't think it's particularly something to do with a RHEL 6.0 - 6.2 compatibility, as things are broking consistently within 6.2.

Let me know if thwere's anything else I can do to debug.

4.8GB tarball source? Ouch. That's actually larger than rpm can currently support, the limit for single file in a package is UINT32_MAX bytes (due to the underlying cpio format limitation). This is checked for files in binary packages but source packages get generated a bit differently and that check, along with various other "large package" aspects is missing for src.rpm's :-/

So the actual bug here is that rpm should've never allowed packaging that thing in the first place, and failure to install/extract is in fact "expected." The uint32_t type used for file sizes gets truncated or wraps around at build-time, causing install/extract to see a shorter file than it is and things go downhill from there.

I fail to see how the src.rpm could've been successfully installed on rhel-6.0. Perhaps you never actually installed the src.rpm on rhel-6.0 either, only (re)built from a spec?

There are (at least) two things needing fixing here, devel_ack for these:
1) Add checks for the maximum allowed file size for sources and patches in src.rpm
2) Fix (or rather, add) "large package" support for src.rpm's as well similar to binary packages (ie while individual file limit is 4GB, the total archive can be much larger but it needs special tracking which is currently missing for src.rpm)

Neither of which is going to directly fix your issue (the cpio format limit will almost certainly not be fixed in rhel-6 or even -7), but its possible to work around it by splitting the sources into max 4GB pieces. That will probably "work" even now, although there will be anomalies with reported total archive sizes and such.

Hm. Right, thanks for taking the time to explain that. Makes sense.

It would be nice to see those enhancements land in RPM at some point though - the use case here is deploying a proprietory program (Matlab) as an RPM by simply bundling the whole thing up, and running the matlab installer in %install. Being able to deply binary software like this as an rpm is a great help when deploying many machines using tooling that makes use of yum repos (katello, pulp, foreman etc).

Back when I was on RHEL 6.0 I was building the SRPM and then successfully generating binary rpms from that SRPM using mock. So it was definitely unpacking OK. However, I will go back and check the file size carefully. The newest matlab is 4.8 GB, perhaps the old one was less than 4.0 GB. Still, I wonder why it won't unpack on 6.2, when it seemed to be doing so on 6.0. Anyway, will check more carefully and see if i can make sense of it.

Thanks again for taking the time to explain the issues. I may even see if I can work up a patch for rpm to at least have it refuse to build broken srpms :)

This request was not resolved in time for the current release.
Red Hat invites you to ask your support representative to
propose this request, if still desired, for consideration in
the next release of Red Hat Enterprise Linux.

This request was erroneously removed from consideration in Red Hat Enterprise Linux 6.4, which is currently under development.  This request will be evaluated for inclusion in Red Hat Enterprise Linux 6.4.

This request was evaluated by Red Hat Product Management for
inclusion in a Red Hat Enterprise Linux release.  Product
Management has requested further review of this request by
Red Hat Engineering, for potential inclusion in a Red Hat
Enterprise Linux release for currently deployed products.
This request is not yet committed for inclusion in a release.

This request was not resolved in time for the current release.
Red Hat invites you to ask your support representative to
propose this request, if still desired, for consideration in
the next release of Red Hat Enterprise Linux.

This request was evaluated by Red Hat Product Management for
inclusion in a Red Hat Enterprise Linux release.  Product
Management has requested further review of this request by
Red Hat Engineering, for potential inclusion in a Red Hat
Enterprise Linux release for currently deployed products.
This request is not yet committed for inclusion in a release.

This request was not resolved in time for the current release.
Red Hat invites you to ask your support representative to
propose this request, if still desired, for consideration in
the next release of Red Hat Enterprise Linux.

Support for files with > 4GB was added upstream but the changes are much too massive to port them back to an already released RHEL version.

We still should issue an error so such broken packages cannot be build in the first place. So devel-ack purely for adding this check.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-1452.html

I Could someone help here to fix the same kind of issue related to Xilinx-SDK

[root@euca-10-254-169-180 SRPMS]# rpm -ivh xilinx-sdk_v20164-2016.4-1.src.rpm
   1: xilinx-sdk_v2016########################################### [100%]
error: unpacking of archive failed on file /root/rpmbuild/SOURCES/xilinx-sdk.tar.gz;58ac1c7f: cpio: Digest mismatch
error: xilinx-sdk_v20164-2016.4-1.src.rpm cannot be installed
[root@euca-10-254-169-180 SRPMS]#

Here the size of the source file is of 14GB and any idea how i can have a workaround for this issue ?

*** Bug 1425417 has been marked as a duplicate of this bug. ***

The workaround is to split the source tarball up into < 4GB chunks and catenate them back in %prep before unpacking. Whether it's really worth the trouble is another question, it's rather desperate.

Oh and for the record, support for files larger than 4GB in the payload was added in rpm 4.12 so it's not in RHEL 7 either, but current Fedora versions do.

Hi Guys,

Thanks for the input here ,
May i know how i can have the %prep section if my source files are splitted into 5 sub-tar balls.
Source6: xilinx-sdk-1.tar.gz
Source7: xilinx-sdk-2.tar.gz
Source8: xilinx-sdk-3.tar.gz
Source9: xilinx-sdk-4.tar.gz
Source10: xilinx-sdk-5.tar.gz

%prep
??
??

Any idea ? Can someone help here pls.

%prep
cat xilinx-sdk-*.tar.gz > %SOURCE0 # if 'Source0:  xilinx-sdk.tar.gz'
%setup ...

Btw:
$ split --verbose -b ... xilinx-sdk.tar.gz xilinx-sdk.tar.gz.'
$ truncate -s 0 xilinx-sdk.tar.gz

Would allow you to do
%prep
cat %SOURCE0.* > %SOURCE0

Do we have to add the whole commands you printed here or just we need to add the :
prep
cat xilinx-sdk-*.tar.gz > %SOURCE0 # if 'Source0:  xilinx-sdk.tar.gz'
%setup ...

1) query is that , how could we use the %setup command  without any options.

Would you be able to provide little more clarity on this pls.