Bug 466548 - posix_fallocate not enabled in libtorrent
Summary: posix_fallocate not enabled in libtorrent
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: libtorrent
Version: 13
Hardware: All
OS: Linux
medium
low
Target Milestone: ---
Assignee: Michel Lind
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2008-10-10 20:43 UTC by Sami Farin
Modified: 2010-10-28 06:03 UTC (History)
3 users (show)

Fixed In Version: rtorrent-0.8.6-4.fc14
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2010-10-27 22:37:39 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
force call to posix_fallocate when creating a new file (536 bytes, patch)
2008-10-10 22:16 UTC, Sami Farin
no flags Details | Diff

Description Sami Farin 2008-10-10 20:43:03 UTC
Description of problem:
posix_fallocate not enabled.
files generated by e.g. rtorrent are horribly fragmented.

Version-Release number of selected component (if applicable):


How reproducible:
always

Steps to Reproduce:
1. download file with e.g. rtorrent
2. type "filefrag importantmovie.avi"
3.
  
Actual results:
66666 extents found

Expected results:
1 extent found

Additional info:
(lazy Friday)

--- libtorrent.spec~    2008-08-07 20:20:46.000000000 +0300
+++ libtorrent.spec     2008-10-10 23:28:51.105802467 +0300
@@ -37,7 +37,7 @@
 export CFLAGS=$RPM_OPT_FLAGS
 export CXXFLAGS=$RPM_OPT_FLAGS
 
-%configure
+%configure --with-posix-fallocate
 make %{?_smp_mflags}
 
 %install

Comment 1 Sami Farin 2008-10-10 22:14:55 UTC
It wasn't as easy as hoped...
But with the attached patch rtorrent successfully calls fallocate as shown by strace, du shows the space is allocated, and filefrag shows 1 extent (at least quite often ;) ).

Comment 2 Sami Farin 2008-10-10 22:16:37 UTC
Created attachment 320068 [details]
force call to posix_fallocate when creating a new file

Comment 3 Bug Zapper 2008-11-26 03:45:44 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 10 development cycle.
Changing version to '10'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 4 Bug Zapper 2009-11-18 09:17:25 UTC
This message is a reminder that Fedora 10 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 10.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '10'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 10's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 10 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 5 Eric Sandeen 2009-11-18 18:16:03 UTC
rawhide at least now does:

File::resize_file() {

...

  if (!SocketFile(m_fd).set_size(m_size))
    return false;

  if (m_flags & flag_fallocate)
    SocketFile(m_fd).reserve(0, m_size);

...
}

so I think we've got the smarts in there.  But configure is still not called w/ --with-posix-fallocate.  Can the original reporter test & confirm that the config option is all we need now?

Can the package owner chime in? :)  We should get this enabled IMHO.

Thanks,
-Eric

Comment 6 Conrad Meyer 2009-11-18 20:49:29 UTC
I guess I acquired the package sometime after this bug was first reported, I don't remember getting mail about it. Anyways, I'm happy to enable such a configure option, if it doesn't require a patch to upstream.

Comment 7 Eric Sandeen 2009-11-18 21:00:06 UTC
I'm not all that familiar w/ rtorrent/libtorrent - in quick testing just the config option didn't make a difference, there was still no call to posix_fallocate.

But I've seen references to rc file setting to enable fallocate as well ... if you know how to actually enable the call to posix_fallocate, maybe you can post it here :)

Comment 8 Conrad Meyer 2009-11-18 21:11:27 UTC
I don't, sorry.

Comment 9 Conrad Meyer 2009-11-18 21:13:02 UTC
Possibly relevant is this:

http://bugs.gentoo.org/226077

The gentoo maintainer's line of thought makes sense to me.

Comment 10 Eric Sandeen 2009-11-18 21:18:46 UTC
Right, it can be painful on, say, ext3, but I was under the impression that it could be controlled by a config file.  *shrug*

Comment 11 Conrad Meyer 2009-11-18 21:25:11 UTC
That's what I got from reading the gentoo bug. If users are on ext4/XFS, they can enable it in their config. I think that at least a large percentage of users are on ext3, and we shouldn't enable it by default.

Comment 12 Eric Sandeen 2009-11-18 21:34:45 UTC
That's fine to leave it disabled in config by default (although, ext4 is now the default fs, but... that's fine...) - point being,

a) enabling it in config doesn't work until we pass the config option anyway, and
b) how do we pass that config option to test it? :)

-Eric

Comment 13 Conrad Meyer 2009-11-18 21:45:23 UTC
Re: a)
Ahhhhh, ok, I misunderstood. My understanding was that there was no such config option, and it was enabled purely at run-time based on the config option.

Re: b)
"To try it out, configure with '--with-posix-fallocate' and set 'system.file_allocate.set = yes'." [0]

But this refers to svn head 6 months ago -- I don't know if we have it in 0.12.5.

[0]: http://libtorrent.rakshasa.no/ticket/460 (about 5/6 of the way down the page)

Comment 14 Bug Zapper 2010-03-15 12:06:47 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 13 development cycle.
Changing version to '13'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 15 Michel Lind 2010-10-15 07:29:26 UTC
(In reply to comment #13)
> Re: a)
> Ahhhhh, ok, I misunderstood. My understanding was that there was no such config
> option, and it was enabled purely at run-time based on the config option.
> 
> Re: b)
> "To try it out, configure with '--with-posix-fallocate' and set
> 'system.file_allocate.set = yes'." [0]
> 
> But this refers to svn head 6 months ago -- I don't know if we have it in
> 0.12.5.

A couple of month after your comment, a comment on the ticket explicitly listed support in 0.12.5:

http://libtorrent.rakshasa.no/ticket/460#comment:46

On the other hand, people are reporting incorrect behavior on ext4. Should this be enabled anyway, and users can decide whether to switch it on in their rtorrent configuration depending on which filesystem they use?

e.g. use it on ext3, probably not on ext4, and use it on btrfs. I'll build a patched libtorrent, but not push it to Bodhi, so we can test this.

Comment 16 Michel Lind 2010-10-15 08:36:08 UTC
Tested the patch and with --with-posix-fallocate and system.file_allocate.set = yes the number of extents created is hugely reduced on ext4. It looks safe to thus push out the changes.

Thanks to delta RPMs, making patches smaller, I'll also push out a new rtorrent build that has the system.file_allocate.set added to the sample configuration file

Comment 17 Fedora Update System 2010-10-15 09:15:05 UTC
rtorrent-0.8.6-4.fc13,libtorrent-0.12.6-2.fc13 has been submitted as an update for Fedora 13.
https://admin.fedoraproject.org/updates/rtorrent-0.8.6-4.fc13,libtorrent-0.12.6-2.fc13

Comment 18 Fedora Update System 2010-10-15 09:15:12 UTC
rtorrent-0.8.6-4.fc14,libtorrent-0.12.6-2.fc14 has been submitted as an update for Fedora 14.
https://admin.fedoraproject.org/updates/rtorrent-0.8.6-4.fc14,libtorrent-0.12.6-2.fc14

Comment 19 Eric Sandeen 2010-10-15 14:51:11 UTC
(In reply to comment #15)

> On the other hand, people are reporting incorrect behavior on ext4. Should this
> be enabled anyway, and users can decide whether to switch it on in their
> rtorrent configuration depending on which filesystem they use?

Out of curiosity what kind of incorrect behavior?

Thanks,
-Eric

Comment 20 Fedora Update System 2010-10-15 19:06:42 UTC
rtorrent-0.8.6-4.fc14, libtorrent-0.12.6-2.fc14 has been pushed to the Fedora 14 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update rtorrent libtorrent'.  You can provide feedback for this update here: https://admin.fedoraproject.org/updates/rtorrent-0.8.6-4.fc14,libtorrent-0.12.6-2.fc14

Comment 21 Fedora Update System 2010-10-27 22:37:33 UTC
rtorrent-0.8.6-4.fc13, libtorrent-0.12.6-2.fc13 has been pushed to the Fedora 13 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 22 Fedora Update System 2010-10-28 06:03:38 UTC
rtorrent-0.8.6-4.fc14, libtorrent-0.12.6-2.fc14 has been pushed to the Fedora 14 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.