Bug 466548 - posix_fallocate not enabled in libtorrent
posix_fallocate not enabled in libtorrent
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: libtorrent (Show other bugs)
13
All Linux
medium Severity low
: ---
: ---
Assigned To: Michel Alexandre Salim
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2008-10-10 16:43 EDT by Sami Farin
Modified: 2010-10-28 02:03 EDT (History)
3 users (show)

See Also:
Fixed In Version: rtorrent-0.8.6-4.fc14
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2010-10-27 18:37:39 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
force call to posix_fallocate when creating a new file (536 bytes, patch)
2008-10-10 18:16 EDT, Sami Farin
no flags Details | Diff

  None (edit)
Description Sami Farin 2008-10-10 16:43:03 EDT
Description of problem:
posix_fallocate not enabled.
files generated by e.g. rtorrent are horribly fragmented.

Version-Release number of selected component (if applicable):


How reproducible:
always

Steps to Reproduce:
1. download file with e.g. rtorrent
2. type "filefrag importantmovie.avi"
3.
  
Actual results:
66666 extents found

Expected results:
1 extent found

Additional info:
(lazy Friday)

--- libtorrent.spec~    2008-08-07 20:20:46.000000000 +0300
+++ libtorrent.spec     2008-10-10 23:28:51.105802467 +0300
@@ -37,7 +37,7 @@
 export CFLAGS=$RPM_OPT_FLAGS
 export CXXFLAGS=$RPM_OPT_FLAGS
 
-%configure
+%configure --with-posix-fallocate
 make %{?_smp_mflags}
 
 %install
Comment 1 Sami Farin 2008-10-10 18:14:55 EDT
It wasn't as easy as hoped...
But with the attached patch rtorrent successfully calls fallocate as shown by strace, du shows the space is allocated, and filefrag shows 1 extent (at least quite often ;) ).
Comment 2 Sami Farin 2008-10-10 18:16:37 EDT
Created attachment 320068 [details]
force call to posix_fallocate when creating a new file
Comment 3 Bug Zapper 2008-11-25 22:45:44 EST
This bug appears to have been reported against 'rawhide' during the Fedora 10 development cycle.
Changing version to '10'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 4 Bug Zapper 2009-11-18 04:17:25 EST
This message is a reminder that Fedora 10 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 10.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '10'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 10's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 10 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 5 Eric Sandeen 2009-11-18 13:16:03 EST
rawhide at least now does:

File::resize_file() {

...

  if (!SocketFile(m_fd).set_size(m_size))
    return false;

  if (m_flags & flag_fallocate)
    SocketFile(m_fd).reserve(0, m_size);

...
}

so I think we've got the smarts in there.  But configure is still not called w/ --with-posix-fallocate.  Can the original reporter test & confirm that the config option is all we need now?

Can the package owner chime in? :)  We should get this enabled IMHO.

Thanks,
-Eric
Comment 6 Conrad Meyer 2009-11-18 15:49:29 EST
I guess I acquired the package sometime after this bug was first reported, I don't remember getting mail about it. Anyways, I'm happy to enable such a configure option, if it doesn't require a patch to upstream.
Comment 7 Eric Sandeen 2009-11-18 16:00:06 EST
I'm not all that familiar w/ rtorrent/libtorrent - in quick testing just the config option didn't make a difference, there was still no call to posix_fallocate.

But I've seen references to rc file setting to enable fallocate as well ... if you know how to actually enable the call to posix_fallocate, maybe you can post it here :)
Comment 8 Conrad Meyer 2009-11-18 16:11:27 EST
I don't, sorry.
Comment 9 Conrad Meyer 2009-11-18 16:13:02 EST
Possibly relevant is this:

http://bugs.gentoo.org/226077

The gentoo maintainer's line of thought makes sense to me.
Comment 10 Eric Sandeen 2009-11-18 16:18:46 EST
Right, it can be painful on, say, ext3, but I was under the impression that it could be controlled by a config file.  *shrug*
Comment 11 Conrad Meyer 2009-11-18 16:25:11 EST
That's what I got from reading the gentoo bug. If users are on ext4/XFS, they can enable it in their config. I think that at least a large percentage of users are on ext3, and we shouldn't enable it by default.
Comment 12 Eric Sandeen 2009-11-18 16:34:45 EST
That's fine to leave it disabled in config by default (although, ext4 is now the default fs, but... that's fine...) - point being,

a) enabling it in config doesn't work until we pass the config option anyway, and
b) how do we pass that config option to test it? :)

-Eric
Comment 13 Conrad Meyer 2009-11-18 16:45:23 EST
Re: a)
Ahhhhh, ok, I misunderstood. My understanding was that there was no such config option, and it was enabled purely at run-time based on the config option.

Re: b)
"To try it out, configure with '--with-posix-fallocate' and set 'system.file_allocate.set = yes'." [0]

But this refers to svn head 6 months ago -- I don't know if we have it in 0.12.5.

[0]: http://libtorrent.rakshasa.no/ticket/460 (about 5/6 of the way down the page)
Comment 14 Bug Zapper 2010-03-15 08:06:47 EDT
This bug appears to have been reported against 'rawhide' during the Fedora 13 development cycle.
Changing version to '13'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 15 Michel Alexandre Salim 2010-10-15 03:29:26 EDT
(In reply to comment #13)
> Re: a)
> Ahhhhh, ok, I misunderstood. My understanding was that there was no such config
> option, and it was enabled purely at run-time based on the config option.
> 
> Re: b)
> "To try it out, configure with '--with-posix-fallocate' and set
> 'system.file_allocate.set = yes'." [0]
> 
> But this refers to svn head 6 months ago -- I don't know if we have it in
> 0.12.5.

A couple of month after your comment, a comment on the ticket explicitly listed support in 0.12.5:

http://libtorrent.rakshasa.no/ticket/460#comment:46

On the other hand, people are reporting incorrect behavior on ext4. Should this be enabled anyway, and users can decide whether to switch it on in their rtorrent configuration depending on which filesystem they use?

e.g. use it on ext3, probably not on ext4, and use it on btrfs. I'll build a patched libtorrent, but not push it to Bodhi, so we can test this.
Comment 16 Michel Alexandre Salim 2010-10-15 04:36:08 EDT
Tested the patch and with --with-posix-fallocate and system.file_allocate.set = yes the number of extents created is hugely reduced on ext4. It looks safe to thus push out the changes.

Thanks to delta RPMs, making patches smaller, I'll also push out a new rtorrent build that has the system.file_allocate.set added to the sample configuration file
Comment 17 Fedora Update System 2010-10-15 05:15:05 EDT
rtorrent-0.8.6-4.fc13,libtorrent-0.12.6-2.fc13 has been submitted as an update for Fedora 13.
https://admin.fedoraproject.org/updates/rtorrent-0.8.6-4.fc13,libtorrent-0.12.6-2.fc13
Comment 18 Fedora Update System 2010-10-15 05:15:12 EDT
rtorrent-0.8.6-4.fc14,libtorrent-0.12.6-2.fc14 has been submitted as an update for Fedora 14.
https://admin.fedoraproject.org/updates/rtorrent-0.8.6-4.fc14,libtorrent-0.12.6-2.fc14
Comment 19 Eric Sandeen 2010-10-15 10:51:11 EDT
(In reply to comment #15)

> On the other hand, people are reporting incorrect behavior on ext4. Should this
> be enabled anyway, and users can decide whether to switch it on in their
> rtorrent configuration depending on which filesystem they use?

Out of curiosity what kind of incorrect behavior?

Thanks,
-Eric
Comment 20 Fedora Update System 2010-10-15 15:06:42 EDT
rtorrent-0.8.6-4.fc14, libtorrent-0.12.6-2.fc14 has been pushed to the Fedora 14 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update rtorrent libtorrent'.  You can provide feedback for this update here: https://admin.fedoraproject.org/updates/rtorrent-0.8.6-4.fc14,libtorrent-0.12.6-2.fc14
Comment 21 Fedora Update System 2010-10-27 18:37:33 EDT
rtorrent-0.8.6-4.fc13, libtorrent-0.12.6-2.fc13 has been pushed to the Fedora 13 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 22 Fedora Update System 2010-10-28 02:03:38 EDT
rtorrent-0.8.6-4.fc14, libtorrent-0.12.6-2.fc14 has been pushed to the Fedora 14 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.