Bug 600663 - libvpx.so.0 fails to load on low-memory machines due to ridiculously oversized .bss
libvpx.so.0 fails to load on low-memory machines due to ridiculously oversize...
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: libvpx (Show other bugs)
13
i686 Linux
low Severity medium
: ---
: ---
Assigned To: Tom "spot" Callaway
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2010-06-05 09:48 EDT by Cesar Eduardo Barros
Modified: 2010-07-15 17:04 EDT (History)
1 user (show)

See Also:
Fixed In Version: libvpx-0.9.1-2.fc12
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2010-07-07 13:41:55 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Cesar Eduardo Barros 2010-06-05 09:48:10 EDT
User-Agent:       Mozilla/5.0 (X11; U; Linux i686; pt-BR; rv:1.9.2.3) Gecko/20100403 Fedora/3.6.3-4.fc13 Firefox/3.6.3

While investigating why playing a sample webm video did not work on my netbook (totem asked for the video/x-vp8 codec, which PackageKit failed to find), I found out the following:

$ rm .gstreamer-0.10/registry.i386.bin
$ gst-inspect | fgrep vp8

(gst-plugin-scanner:2559): GStreamer-WARNING **: Failed to load plugin '/usr/lib/gstreamer-0.10/libgstvp8.so': libvpx.so.0: cannot map zero-fill pages: Não foi possível alocar memória

A strace gave me the following information:

$ strace -f -o /tmp/vpx.strace gst-inspect >/dev/null

[...]
2563  open("/usr/lib/libvpx.so.0", O_RDONLY) = 7
2563  read(7, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\340\"\0\0004\0\0\0"..., 512) = 512
2563  fstat64(7, {st_mode=S_IFREG|0755, st_size=409800, ...}) = 0
2563  mmap2(NULL, 540003044, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 7, 0) = 0x972c5000
2563  mmap2(0x97328000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 7, 0x63) = 0x97328000
2563  mmap2(0x97329000, 539593444, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
2563  close(7)                          = 0
[...]

Clearly, it is trying to allocate an absurdly high amount of memory (over 512M, which is more than this netbook has).

objdump tells the rest of the history:

$ objdump -x /usr/lib/libvpx.so.0
[...]
Program Header:
    LOAD off    0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**12
         filesz 0x00062568 memsz 0x00062568 flags r-x
    LOAD off    0x00063000 vaddr 0x00063000 paddr 0x00063000 align 2**12
         filesz 0x00000ad0 memsz 0x20299ae4 flags rw-
[...]
Sections:
Idx Name          Size      VMA       LMA       File off  Algn
[...]
 23 .bss          20299004  00063ae0  00063ae0  00063ad0  2**5
                  ALLOC
[...]

The .bss is clearly way too huge.

This machine has only 512M of RAM, and no swap. /proc/sys/vm/overcommit_memory is at its default of 0.

This could cause problems on 32-bit machines, even with more memory, since 512M is a fair chunk of the address space. If it is fragmented enough, libvpx will fail to load (on 64-bit, the address space is large enough that this should not be a problem).

Reproducible: Always

Steps to Reproduce:
1. Configure a 32-bit machine with 512M of RAM, no swap, and vm.overcommit_memory=0
2. Erase ~/.gstreamer-0.10/registry.i386.bin
3. Run gst-inspect
Actual Results:  
Loading libvpx.so.0 fails with "cannot map zero-fill pages"

Expected Results:  
libvpx.so.0 loads correctly, without wasting a huge chunk of the available address space
Comment 1 Cesar Eduardo Barros 2010-06-05 10:08:45 EDT
The culprit seems to be vp8/encoder/ssim.c:

[...]
static short img12_sum_block[8*4096*4096*2] ;

static short img1_sum[8*4096*2];
static short img2_sum[8*4096*2];
static int   img1_sq_sum[8*4096*2];
static int   img2_sq_sum[8*4096*2];
static int   img12_mul_sum[8*4096*2];
[...]

(from upstream's git repository)
Comment 2 Tom "spot" Callaway 2010-06-11 12:22:58 EDT
Opened upstream as: http://code.google.com/p/webm/issues/detail?id=78
Comment 3 Cesar Eduardo Barros 2010-06-13 12:34:27 EDT
Probably fixed upstream by commit 9099fc0d69a69525656b4bfeeb1e7aabec04897b (I have not tested it).
Comment 4 Cesar Eduardo Barros 2010-06-13 12:39:11 EDT
Related Debian bug: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=583765 (found at the comments at http://review.webmproject.org/#change,136). According to it, it seems this bug could also cause some problems with prelink.
Comment 5 Fedora Update System 2010-06-28 14:57:49 EDT
libvpx-0.9.1-2.fc12 has been submitted as an update for Fedora 12.
http://admin.fedoraproject.org/updates/libvpx-0.9.1-2.fc12
Comment 6 Fedora Update System 2010-06-28 14:58:59 EDT
libvpx-0.9.1-2.fc13 has been submitted as an update for Fedora 13.
http://admin.fedoraproject.org/updates/libvpx-0.9.1-2.fc13
Comment 7 Fedora Update System 2010-06-28 19:37:04 EDT
libvpx-0.9.1-2.fc13 has been pushed to the Fedora 13 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update libvpx'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/libvpx-0.9.1-2.fc13
Comment 8 Fedora Update System 2010-06-29 11:30:28 EDT
libvpx-0.9.1-2.fc12 has been pushed to the Fedora 12 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update libvpx'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/libvpx-0.9.1-2.fc12
Comment 9 Cesar Eduardo Barros 2010-06-29 20:04:01 EDT
libvpx-0.9.1-2.fc13 works for me in the low-memory machine mentioned in this bug report. I had to remove .gstreamer-0.10/registry.i386.bin to make it register, probably because none of the gstreamer libraries had changed its mtime:

$ gst-inspect | fgrep vp8
$ rm .gstreamer-0.10/registry.i386.bin 
$ gst-inspect | fgrep vp8
vp8:  vp8enc: On2 VP8 Encoder
vp8:  vp8dec: On2 VP8 Decoder
$ 

The .bss, while still very large, is no long ridiculously so:

 23 .bss          00198fe4  00061b40  00061b40  00060b30  2**5
                  ALLOC

That is around 1.5M BSS, much better than the more than 512M it was before.

The only problem left seems to be that the .webm extension does not seem to be associated with totem, but that would be a separate bug report.
Comment 10 Fedora Update System 2010-07-07 13:41:44 EDT
libvpx-0.9.1-2.fc13 has been pushed to the Fedora 13 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 11 Fedora Update System 2010-07-15 17:04:32 EDT
libvpx-0.9.1-2.fc12 has been pushed to the Fedora 12 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.