Bug 516309 - tar fails (on large files >8GB?) with value # out of off_t range 0..8589934591 error
Summary: tar fails (on large files >8GB?) with value # out of off_t range 0..858993459...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: tar
Version: rawhide
Hardware: All
OS: Linux
low
medium
Target Milestone: ---
Assignee: Pavel Raiskup
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 517000 1334559 1347229
TreeView+ depends on / blocked
 
Reported: 2009-08-07 23:17 UTC by Orion Poplawski
Modified: 2016-06-16 10:44 UTC (History)
5 users (show)

Fixed In Version:
Clone Of:
: 1334559 (view as bug list)
Environment:
Last Closed: 2013-03-01 11:50:01 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Orion Poplawski 2009-08-07 23:17:33 UTC
Description of problem:

tar is failing as part of an amanda backup run with:

1249611157.184389: amgtar: Spawning "/bin/tar /bin/tar --create --file /dev/null --directory /export/backup1/paiute --one-file-system --atime-preserve=system --no-check-device --listed-incremental /var/lib/amanda/gnutar-lists/saga_export_backup1_paiute_0.new --sparse --xattrs --ignore-failed-read --totals --exclude-from /var/log/amanda/amgtar._export_backup1_paiute.20090806201237.exclude ." in pipeline
1249611673.453231: amgtar: /bin/tar: value 11318425088 out of off_t range 0..8589934591
1249612678.826360: amgtar: Total bytes written: 184536729600 (172GiB, 116MiB/s)
1249612678.869487: amgtar: /bin/tar: Exiting with failure status due to previous errors         
1249612678.869542: amgtar: .....
1249612678.869603: amgtar: estimate time for /export/backup1/paiute level 0: 1521.685
1249612678.869623: amgtar: estimate size for /export/backup1/paiute level 0: 180211650 KB
1249612678.869651: amgtar: waiting for /bin/tar "/export/backup1/paiute" child
1249678.870126: amgtar: after /bin/tar /export/backup1/paiute wait
1249612678.870163: amgtar: /bin/tar exited with status 2: see /var/log/amanda/client/Data/amgtar.20090806201237.debug

Version-Release number of selected component (if applicable):
tar-1.22-4.fc11.i586

Seems to be one of the options triggering, just tarring up the file does not produce the error.  

After some more testing, looks like it is --xattrs that triggers it.

Comment 1 Ondrej Vasik 2009-08-09 18:57:51 UTC
Thanks for report, will check our xattrs patch - it's quite possible that this patch could be culprit of that limitation.

Comment 2 Orion Poplawski 2009-08-18 14:04:17 UTC
Any ETA on a fix - this is really causing problems with amanda backups.

Comment 3 Ondrej Vasik 2009-08-18 15:08:10 UTC
I did just a quick review of the patch and found nothing obvious. It's on top of my todo list at the moment, so I hope I'll find something soon. I'll try to reproduce it myself on my machine with some smaller reproducer.

Comment 4 Ondrej Vasik 2009-08-20 11:46:09 UTC
Tried to reproduce with 100 files with size of 8,5G (8912896000 bytes) with extended attributes (--dereference option just because they are symlinks ;) ) and with "tar --create --file /dev/null --directory . --one-file-system --atime-preserve=system --no-check-device --sparse --xattrs --listed-incremental temp.new --ignore-failed-read --totals --exclude-from temp.exclude --dereference mybigfile.*
" , but the error doesn't seem to occur then (just passes with Total bytes written: 900202659840 (839GiB, 76GiB/s)) on my i386 architecture.

Do you have some simple reproducer? Some specific filesystem or mount options?

Comment 5 Orion Poplawski 2009-08-20 15:30:20 UTC
Looks like it might be interaction with --sparse and an actual (or at least apparently) sparse file.  I've posted my reproducing file (20GB) here:

http://www.cora.nwra.com/~orion/paperSin38_3_g01.cdf

gdb seems to indicate that it may be using the old gnu format to dump the sparse file and maybe that's the issue?

(gdb) bt
#0  to_chars (negative=<value optimized out>, value=0, valsize=8, substitute=0, where=0x80923ca "07143137000",
    size=12, type=0x807cadd "off_t") at create.c:353
#1  0x0805093b in off_to_chars (v=965524992, p=0x80923ca "07143137000", s=12) at create.c:433
#2  0x080636c1 in oldgnu_store_sparse_info (file=0xbfffeefc, pindex=0xbfffeeac, sp=0x80923ca, sparse_size=1)
    at sparse.c:696
#3  0x080637d2 in oldgnu_dump_header (file=0xbfffeefc) at sparse.c:721
#4  0x08063f09 in tar_sparse_dump_header (file=<value optimized out>) at sparse.c:154
#5  sparse_dump_file (file=<value optimized out>) at sparse.c:395
#6  0x0805252a in dump_file0 (st=0xbffff098, p=<value optimized out>, top_level=<value optimized out>,
    parent_device=2193) at create.c:1643
#7  0x08053521 in dump_file (p=0x80918d8 "./paperSin38_3_g01.cdf", top_level=0, parent_device=1)
    at create.c:1824
#8  0x080531ba in dump_dir0 (parent_device=<value optimized out>, top_level=<value optimized out>,
    st=<value optimized out>, directory=<value optimized out>) at create.c:1244
#9  dump_dir (parent_device=<value optimized out>, top_level=<value optimized out>, st=<value optimized out>,
    directory=<value optimized out>) at create.c:1293
#10 dump_file0 (parent_device=<value optimized out>, top_level=<value optimized out>,
    st=<value optimized out>, directory=<value optimized out>) at create.c:1631
#11 0x08053521 in dump_file (p=0x8090e30 ".", top_level=1, parent_device=1) at create.c:1824
#12 0x080535b6 in create_archive () at create.c:1361
#13 0x08066b25 in main (argc=13, argv=0xbffff564) at tar.c:2561

Comment 6 Ondrej Vasik 2009-08-21 12:40:23 UTC
old gnu format and gnu format have the same optab functions in sparse.c, I don't think that's the issue. Thanks for reproducer, but 20G is quite big, I'll try to reproduce it somehow without downloading it.

Comment 7 Orion Poplawski 2009-09-14 22:15:06 UTC
Any idea here?  Anything else I can do to help?

Comment 8 Ondrej Vasik 2009-09-15 05:27:28 UTC
Maybe ... could you try to reproduce this problem with sparse file created via dd? 
I tried several sparse files created by dd(different sizes, different skips/seeks, more "holes"), but I was not able to reproduce this issue. Downloading your reproducer file would mean quite a lot of bandwidth and time (I started download, but with 20kB/s speed it's nonsense to download it) - so I would be happy with sparse reproducer that could be generated somehow on my machine ...

Comment 9 Orion Poplawski 2009-09-15 23:01:07 UTC
I managed to compress the cdf file to 6.5GB.  We should have 3Mbps outgoing barring other usage so you might have more luck.

http://www.cora.nwra.com/~orion/tartest.cdf.bz2

Another possible piece of the puzzle, is that this is a sparse file created by using rsync --sparse.  On the original system, it is not sparse. 

Do you know of any tools to analyze sparse files?  One possibility is to scan the file and print out a list of the holes in order to reproduce it.  Might not be too hard to write in C.

Also trying to create:
http://www.cora.nwra.com/~orion/tartest.cdf.xz

Not done yet so don't know how big that will be yet.

Comment 10 Orion Poplawski 2009-09-16 04:18:34 UTC
xz file is 3.2GB

Comment 11 Ondrej Vasik 2009-09-16 05:49:59 UTC
Ok, started to download xz to my home computer, hopefully it will work with that file. Thanks, I'll let you know later ...

Comment 12 Ondrej Vasik 2009-09-16 20:11:10 UTC
Bad luck with this sparse file ... 

Finally downloaded xz and 
$ xz -d tartest.cdf.xz 
xz: tartest.cdf.xz: Compressed data is corrupt

Old GNU (and POSIX) tar header has 8G limitation, but this should not be default format. Will try non-sparse large >8G file and few other things before another download ...

Comment 13 Orion Poplawski 2009-09-16 20:51:24 UTC
Turns out my original compress failed.  Re-running now but taking a long time.  Will post when it is done.

Comment 14 Orion Poplawski 2009-09-16 22:58:01 UTC
Complete, but 5.4GB now.

Comment 15 Kamil Dudka 2009-11-18 12:27:35 UTC
Any chance the bug is specific for a particular architecture? Can you e.g. compare the behavior on 32bit vs. 64bit arch? And please provide some details about your kernel, filesystem etc. Thanks in advance!

Comment 16 Ondrej Vasik 2010-01-06 19:36:36 UTC
Sorry for late check, Orion, got a bit busy and bought new computer at home - and was not able to access your downloaded file for a while. However - I'm still not able to reproduce even with your file (I have F-12, but I checked F-11 tar sources). 

[Reset@localhost src]$ pwd
/home/Reset/koji/tar/F-11/tar-1.22/src
[Reset@localhost src]$ touch temp.new
[Reset@localhost src]$ touch temp.exclude
[Reset@localhost src]$ ./tar --create --file /dev/null --directory . --one-file-system --atime-preserve=system --no-check-device --sparse --xattrs --listed-incremental temp.new --ignore-failed-read --totals --exclude-from temp.exclude ~/koji/tar/devel/tartest.cdf 
./tar: Removing leading `/' from member names
Total bytes written: 21457561600 (20GiB, 154GiB/s)

Do you still experience the issue? Is there something wrong on the command syntax I used? Could it be arch specific? I have i386 architecture on my machine (but from report it seems you have the same). Are you able to experience that issue directly with tar binary or just within amanda backup (maybe ulimit there?) ?
Thanks in advance for answer...

Comment 17 Orion Poplawski 2010-01-07 18:27:52 UTC
Finally managed to break at the point of error:

#0  to_chars (negative=0, value=11318425088, valsize=8, substitute=0,
    where=0x809407c '0' <repeats 11 times>, size=12, type=0x807ca1d "off_t") at create.c:302
#1  0x0805098c in off_to_chars (v=<value optimized out>, p=<value optimized out>,
    s=<value optimized out>) at create.c:433
#2  0x08062995 in pax_dump_header_1 (file=<value optimized out>) at sparse.c:1033
#3  pax_dump_header (file=<value optimized out>) at sparse.c:1064
#4  0x08064029 in tar_sparse_dump_header (file=<value optimized out>) at sparse.c:154
#5  sparse_dump_file (file=<value optimized out>) at sparse.c:395
#6  0x080525c7 in dump_file0 (st=0xbffff168, p=<value optimized out>,
    top_level=<value optimized out>, parent_device=<value optimized out>) at create.c:1643
#7  0x08053602 in dump_file (p=<value optimized out>, top_level=<value optimized out>,
    parent_device=<value optimized out>) at create.c:1827
#8  0x08053717 in create_archive () at create.c:1320
#9  0x08066c6b in main (argc=<value optimized out>, argv=<value optimized out>) at tar.c:2561

this is now with tar-1.22-10.fc12.i686.

command is:
--create --file /dev/null --directory /data/backup1/paiute/export/home/alexand/COMMAS/alison --one-file-system --atime-preserve=system --no-check-device --listed-incremental /tmp/temp.new --sparse --xattrs --ignore-failed-read --totals --exclude-from /var/log/amanda/amgtar._export_backup1_paiute.20090825161003.exclude paperSin38_3_g01.cdf              

It's trying to store the size of the shrunken file:

sparse.c:1032
  /* Store the effective (shrunken) file size */
  OFF_TO_CHARS (file->stat_info->archive_file_size, blk->header.size);

 and it is larger than 8589934591.

Should give you some more info on how to reproduce.  Any reason why we would be limited to off_t in storing this size when we aren't in other file sizes?

Comment 18 Bug Zapper 2010-04-28 09:37:30 UTC
This message is a reminder that Fedora 11 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 11.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '11'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 11's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 11 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 19 Bug Zapper 2010-07-30 10:42:45 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 14 development cycle.
Changing version to '14'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 20 Ondrej Vasik 2010-12-20 13:09:09 UTC
Sorry for long delay, it got off my radar - and current mail on bug-tar mailing lists remind me that's still unresolved.

As the message is print by to_chars_subst(), maxval in the error message is clear that gnu_format variable is not set - so the archive_format global variable is not GNU_FORMAT or OLDGNU_FORMAT - so maxval limits the filesize to 8GB. However - POSIX_FORMAT should not be limited that way.
As extended attributes usage forces POSIX_FORMAT (and usual default format is GNU), that's maybe the culprit why --xattrs causes the issue...

Comment 21 Bert DeKnuydt 2012-09-19 10:19:57 UTC
This is still present in tar-1.26-7.fc17.x86_64 when both --acls and --sparse are used.

Comment 22 Tramada 2012-09-27 03:01:27 UTC
Also get the same error on EL6 (tar-1.23-7.el6.x86_64).
There is much simpler command to reproduce it.

$ tar -co 24GBtestfile >/dev/null
tar: value 24104711334 out of off_t range 0..8589934591
tar: Exiting with failure status due to previous errors

The same command works just fine for files smaller then 8589934591 bytes.
Thankfully the old fashion syntax works without a fail.

$ tar -cf - 24GBtestfile >/dev/null
$

Comment 23 Pavel Raiskup 2013-01-25 18:30:11 UTC
Upstream proposal:

http://www.mail-archive.com/bug-tar@gnu.org/msg03905.html

Comment 24 Pavel Raiskup 2013-01-28 14:28:15 UTC
(In reply to comment #22)
> Also get the same error on EL6 (tar-1.23-7.el6.x86_64).
> There is much simpler command to reproduce it.
> 
> $ tar -co 24GBtestfile >/dev/null
> tar: value 24104711334 out of off_t range 0..8589934591
> tar: Exiting with failure status due to previous errors
> 
> The same command works just fine for files smaller then 8589934591 bytes.
> Thankfully the old fashion syntax works without a fail.
> 
> $ tar -cf - 24GBtestfile >/dev/null
> $

Hi Tramada, this is not related to this bugzilla.  You have used old 'v7' format
by using the -o option.  Probably misspelled with '-O'.  This legacy format is
expected to not store files > 8GB.

Comment 25 Pavel Raiskup 2013-03-01 09:42:53 UTC
Upstream fixed this issue:

http://lists.gnu.org/archive/html/bug-tar/2013-01/msg00002.html

I'll backport this patch today.  Even if there is broken listing of such
created tarballs - it is another bug.  I'll create new BZ# for this one.

Pavel

Comment 26 Pavel Raiskup 2013-03-01 11:50:01 UTC
> [... add listing ...]
> I'll create new BZ# for this one.

Bug created as bug #916995.

Reported problem from this bugzilla is fixed & built in rawhide.

~> https://koji.fedoraproject.org/koji/taskinfo?taskID=5066463

Pavel

Comment 27 Fedora Update System 2013-03-19 15:08:51 UTC
tar-1.26-12.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/tar-1.26-12.fc18

Comment 28 Fedora Update System 2013-03-24 00:02:51 UTC
tar-1.26-12.fc18 has been pushed to the Fedora 18 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.