Bug 1662481 - rpmdev-extract and rpm2cpio does not work for files greater 4G
Summary: rpmdev-extract and rpm2cpio does not work for files greater 4G
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: rpmdevtools
Version: 35
Hardware: x86_64
OS: Linux
unspecified
low
Target Milestone: ---
Assignee: Neal Gompa
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-12-28 20:43 UTC by Edgar Hoch
Modified: 2022-12-13 15:12 UTC (History)
11 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2022-12-13 15:12:49 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
Description of test examples (3.94 KB, text/plain)
2018-12-28 20:43 UTC, Edgar Hoch
no flags Details
Spec file for size below 4G (230 bytes, text/plain)
2018-12-28 20:44 UTC, Edgar Hoch
no flags Details
Spec file for size above 4G (4.87 MB, application/x-rpm)
2018-12-28 20:45 UTC, Edgar Hoch
no flags Details
Spec file for size above 4G (230 bytes, text/plain)
2018-12-28 20:46 UTC, Edgar Hoch
no flags Details
SRPM for below 4G (3.89 MB, application/x-rpm)
2018-12-28 20:47 UTC, Edgar Hoch
no flags Details
SRPM for above 4G (4.87 MB, application/x-rpm)
2018-12-28 20:47 UTC, Edgar Hoch
no flags Details

Description Edgar Hoch 2018-12-28 20:43:29 UTC
Created attachment 1517283 [details]
Description of test examples

Description of problem:
It seams that if a source rpm file is greater than 4 G, then rpmdev-extract cannot handle such a big file.

Instead of creating a spec file and a tar file (in my test), it expands the tar file contained in the srpm, and doesn't create the spec file.

Please see the attached files for test examples and the results.


Version-Release number of selected component (if applicable):
rpmdevtools-8.10-7.fc29.noarch
rpm-4.14.2.1-2.fc29.x86_64
cpio-2.12-9.fc29.x86_64

How reproducible:
Always


Additional note: I don't now if rpmdev-extract really does the wrong thing, or if something goes wrong already at rpmbuild step.

Comment 1 Edgar Hoch 2018-12-28 20:44:21 UTC
Created attachment 1517284 [details]
Spec file for size below 4G

Comment 2 Edgar Hoch 2018-12-28 20:45:12 UTC
Created attachment 1517297 [details]
Spec file for size above 4G

Comment 3 Edgar Hoch 2018-12-28 20:46:29 UTC
Created attachment 1517298 [details]
Spec file for size above 4G

Comment 4 Edgar Hoch 2018-12-28 20:47:32 UTC
Created attachment 1517299 [details]
SRPM for below 4G

Comment 5 Edgar Hoch 2018-12-28 20:47:57 UTC
Created attachment 1517300 [details]
SRPM for above 4G

Comment 6 Michael Schwendt 2018-12-29 11:07:52 UTC
The underlying command to extra from .rpm archives is "rpm2cpio". See "man rpm2cpio". Here's an example that would extract a single file with the name "somerpm.spec":

  rpm2cpio somerpm.src.rpm | cpio -im somerpm.spec

However, if you do that with your bigtest-above src.rpm, apparently that leads to trouble already, and cpio doesn't see the .tar file, but the files within the .tar file. Using "cpio -t" to list the files in the cpio archive shows that, too:

$ rpmls -p bigrpmtest-above-1-1.fc29.src.rpm 
-rw-rw-r--  bigrpmtest-above-tar.tar
-rw-rw-r--  bigrpmtest-above.spec

$ rpm2cpio bigrpmtest-above-1-1.fc29.src.rpm | cpio -t
cpio: warning: skipped 16 bytes of junk
file0
file1
file2
file3
file4
10240007 blocks

Comment 7 Edgar Hoch 2018-12-29 13:28:34 UTC
Michael, thanks for investigating the problem. Yes, rpm2cpio displays the same messages as rpmdev-extract in this case.

rpm2cpio belongs to package rpm, so I reassign it to rpm.

Comment 8 Panu Matilainen 2019-01-02 09:47:26 UTC
The issue is that there is no cpio format that can handle files > 4GB and upstream is not interested in adding one (we've tried), so there's nothing rpm2cpio can do about it. Except spit out an actual error, which I just added, so thanks for pointing this out:
https://github.com/rpm-software-management/rpm/commit/017d0b8846cdb62d1b6cc3e48d82eae6da2c5b74

Things are different for rpmdev-extract though, since it actually extracts the files onto filesystem rather than spit out a specific format stream. rpmdev-extract just needs to be changed to use rpm2archive (in rpm >= 4.12) + tar instead of rpm2cpio + cpio to handle large files, so handing back to rpmdevtools.

Comment 9 Neal Gompa 2019-03-03 03:23:33 UTC
If this fix to rpm2archive[1] is backported into Fedora stable releases, then I can certainly change rpmdev-extract to use it. Panu, would you be willing to do that?

[1]: https://github.com/rpm-software-management/rpm/commit/ef422951aeb328c3274bc7b355bf8387306590f3

Comment 10 Panu Matilainen 2019-03-04 08:52:26 UTC
That fix is something that should go into the next upstream bugfix release, that's how it'll get into Fedora.

Comment 11 Neal Gompa 2019-03-17 03:30:47 UTC
(In reply to Panu Matilainen from comment #10)
> That fix is something that should go into the next upstream bugfix release,
> that's how it'll get into Fedora.

Then I can't do anything until then, unfortunately.

Comment 12 Edgar Hoch 2019-03-17 04:30:23 UTC
I think, upstream should create a new release soon and as often as there are bug found and fixed (not only once a year).

If upstream is not doing that, or not willing to do it, Fedora should fix it temporary until upstream contains the fix. Either Fedora packages should be build from current git master, or known bugs should be fixed by temporary adding patches downstream to Fedora packages. This is often done in other packages, and I think it is a good methode to solve problems. Preserving bugs in Fedora for a long time is not a good approach.

Sometimes it doesn't seem understandable to me why software developers don't make a new release, at least a bug-fix release, even though bugs are known and solutions exist - often about many bugs and solutions. Is it necessary to have 10 or 100 bugs, isn't one bug enough to be fixed? Of course, one shouldn't release every day, and software should contain as few bugs as possible. But if there are bugs, they should be fixed quickly - not only at the developer's site, but also at the user's site - in this case in the Fedora packages.

Comment 13 Ben Cotton 2019-10-31 19:47:06 UTC
This message is a reminder that Fedora 29 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 29 on 2019-11-26.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '29'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 29 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 14 Ben Cotton 2020-02-11 15:45:54 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 32 development cycle.
Changing version to 32.

Comment 15 Fedora Program Management 2021-04-29 17:08:55 UTC
This message is a reminder that Fedora 32 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 32 on 2021-05-25.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '32'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 32 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 16 Edgar Hoch 2021-04-29 17:44:30 UTC
The problem still exists on Fedora 34.

rpmdevtools-9.3-4.fc34.noarch
rpm-4.16.1.3-1.fc34.x86_64
cpio-2.13-10.fc34.x86_64

Comment 17 Ben Cotton 2021-08-10 13:43:54 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 35 development cycle.
Changing version to 35.

Comment 18 Ben Cotton 2022-11-29 16:46:01 UTC
This message is a reminder that Fedora Linux 35 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 35 on 2022-12-13.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '35'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 35 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Comment 19 Ben Cotton 2022-12-13 15:12:49 UTC
Fedora Linux 35 entered end-of-life (EOL) status on 2022-12-13.

Fedora Linux 35 is no longer maintained, which means that it
will not receive any further security or bug fix updates. As a result we
are closing this bug.

If you can reproduce this bug against a currently maintained version of Fedora Linux
please feel free to reopen this bug against that version. Note that the version
field may be hidden. Click the "Show advanced fields" button if you do not see
the version field.

If you are unable to reopen this bug, please file a new report against an
active release.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.