Bug 1352061

Summary:

btrfs: stat reports the st_blocks with delay (data loss in archivers)

Product:

[Fedora] Fedora

Reporter:

Pavel Raiskup <praiskup>

Component:

kernel

Assignee:

Kernel Maintainer List <kernel-maint>

Status:

CLOSED UPSTREAM

QA Contact:

Fedora Extras Quality Assurance <extras-qa>

Severity:

unspecified

Docs Contact:

Priority:

unspecified

Version:

CC:

aschorr, gansalmon, itamar, jonathan, kdudka, kernel-maint, madhu.chinakonda, mchehab

Target Milestone:

---

Target Release:

---

Hardware:

Unspecified

OS:

Unspecified

Whiteboard:

Fixed In Version:

Doc Type:

If docs needed, set a value

Doc Text:

Story Points:

---

Clone Of:

Environment:

Last Closed:

2016-07-01 15:23:23 UTC

Type:

Bug

Regression:

---

Mount Type:

---

Documentation:

---

CRM:

Verified Versions:

Category:

---

oVirt Team:

---

RHEL 7.3 requirements from Atomic Host:

Cloudforms Team:

---

Target Upstream Version:

Embargoed:

Attachments:

Description	Flags
Reproducer.	none

Description Pavel Raiskup 2016-07-01 14:10:54 UTC

There are optimizations in data archivers (tar, rsync, ...) that rely on
st_blocks info.

It looks like btrfs doesn't show correct value in 'st_blocks' until the data are
synced.  For example, in tar there is optimization checking whether the
'st_size' reports more data than the 'st_blocks' can hold --> then tar considers
that file is sparse and does additional steps.

ATM, there happens that:

    a) some "tool" creates sparse file
    b) that tool does not flush explicitly
    c) tar is called immediately to archive that sparse file
    d) tar considers [1] the file is completely sparse (because st_blocks is
       zero) and archives no data.  Here comes data loss.

Because we fixed 'btrfs' to report non-zero 'st_blocks' when the file data are
in-lined.  I consider this is really bug btrfs worth fixing.

[1] http://git.savannah.gnu.org/cgit/paxutils.git/tree/lib/system.h?id=ec72abd9dd63bbff4534ec77e97b1a6cadfc3cf8#n392

Comment 1 Pavel Raiskup 2016-07-01 14:26:52 UTC

Created attachment 1174941 [details]
Reproducer.

$ make
    gcc main.c -O0 -g3 -o binary
    test `stat --file-system --format '%T' main.c` = btrfs
    while : ; do ./reproducer; done
    stat reported zero blocks!
    still zero blocks
    (mostly?) synced
    stat reported zero blocks!
    (mostly?) synced
    stat reported zero blocks!
    still zero blocks
    still zero blocks
    still zero blocks
    still zero blocks
    still zero blocks
    still zero blocks
    still zero blocks
    still zero blocks
    still zero blocks
    still zero blocks
    still zero blocks
    still zero blocks
    still zero blocks
    still zero blocks
    still zero blocks
    still zero blocks
    still zero blocks
    still zero blocks
    still zero blocks
    still zero blocks
    still zero blocks
    still zero blocks
    still zero blocks
    (mostly?) synced
    ....


Expected output:
    gcc main.c -O0 -g3 -o binary
    test `stat --file-system --format '%T' main.c` = btrfs
    while : ; do ./reproducer; done
    (busy loop here)

Comment 2 Josh Boyer 2016-07-01 15:23:23 UTC

Please report this directly to the upstream btrfs maintainers:

Chris Mason <clm> (maintainer:BTRFS FILE SYSTEM)
Josef Bacik <jbacik> (maintainer:BTRFS FILE SYSTEM)
David Sterba <dsterba> (maintainer:BTRFS FILE SYSTEM)
linux-btrfs.org (open list:BTRFS FILE SYSTEM)
linux-kernel.org (open list)

Comment 3 Pavel Raiskup 2016-07-04 04:34:15 UTC

Upstream thread:
https://mail-archive.com/linux-btrfs@vger.kernel.org/msg55383.html