Bug 720438

sed-4.2.1-7.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/sed-4.2.1-7.fc15

sed-4.2.1-6.fc14 has been submitted as an update for Fedora 14.
https://admin.fedoraproject.org/updates/sed-4.2.1-6.fc14

In my F15 x86_64 host, with 4 GiB of RAM, the update fixes the bug - output before and after updating sed:

[andre@compaq-pc tmp]$ perl -le 'print "v"x(2**31-1)' |sed 's/^/v/' |wc
      0       0       0
[andre@compaq-pc tmp]$ perl -le 'print "v"x(2**31-1)' |sed 's/^/v/' |wc
      1       1 2147483648
[andre@compaq-pc tmp]$

However, in my VirtualBox F14 and F15 x86_64 guests with 1 GiB of RAM allocated to each, it does not - the output is still "      0       0       0". I also noticed that in the guests, the command returns earlier. So I'm guessing that with insufficient RAM, the command exits prematurely giving the wrong result? And if so, which component is responsible?

I believe it is perl, but you can test:

perl -le 'print "v"x(2**31-1)' > file
ls -l file

if empty perl is the culprit.
if not empty sed is the culprit, you can confirm with:

sed 's/^/v/' file | wc

Also important is testing that sed -i does not destroy a file.  In your 1GiB guests, try:

perl -e 'print "v"x(2**29)' > small  # note no -l here
cat small small small small > file
wc file                              # should be 0 1 2147483648
sed -i 's/^/v/' file                 # should fail
wc file                              # should still be 0 1 2147483648

You will need a few GiB of free disk space for this test.  Thanks!

In my F15 x86_64 1 GiB guest:

[andre@localhost ~]$ perl -le 'print "v"x(2**31-1)' > file
[andre@localhost ~]$ ls -l file
-rw-rw-r--. 1 andre andre 2147483648 Jul 13 06:48 file
[andre@localhost ~]$ sed 's/^/v/' file | wc
sed: couldn't re-allocate memory
      0       0       0
[andre@localhost ~]$

and

[andre@localhost ~]$ perl -e 'print "v"x(2**29)' > small
[andre@localhost ~]$ cat small small small small > file
[andre@localhost ~]$ wc file
         0          1 2147483648 file
[andre@localhost ~]$ sed -i 's/^/v/' file
sed: couldn't re-allocate memory
[andre@localhost ~]$ wc file
         0          1 2147483648 file
[andre@localhost ~]$

I'll assume that the F14 behavior is the same, and leave my karma at -1 for both (although if this issue is hard to fix, and existed before, I could change them to +1 anyway since it works for large amounts of RAM).

1) Yes, the issue existed before too.  There's nothing really to fix, perhaps it's just extra swap that you need.

2) The second test is the important one, i.e. sed doesn't destroy data when it exits with an error.

As long as the exit code of sed is 1 after the "couldn't re-allocate memory" error I think the patch is fine.  Before, the failure was silent and happened independent of the amount of memory.

On the same guest, the exit code is 0, not 1:

[andre@localhost ~]$ sed 's/^/v/' file | wc
sed: couldn't re-allocate memory
      0       0       0
[andre@localhost ~]$ echo $?
0
[andre@localhost ~]$

Hold on, that would be the exit code of wc, not sed. Let me figure out how to test sed itself and report back.

[andre@localhost ~]$ sed 's/^/v/' file > file2
sed: couldn't re-allocate memory
[andre@localhost ~]$ echo $?
4
[andre@localhost ~]$

I'll change my F15 karma to +1, and retest F14 as well.

Thanks!

sed-4.2.1-7.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.

sed-4.2.1-6.fc14 has been pushed to the Fedora 14 stable repository.  If problems still persist, please make note of it in this bug report.