Bug 1268347

Clone for RHEL6 (not tested)

+++ This bug was initially created as a clone of Bug #1265196 +++

Description of problem:

On an NFS mount point:

$ qemu-img create -f qcow2 -o preallocation=falloc disk.img 262144
Formatting 'disk.img', fmt=qcow2 size=262144 encryption=off cluster_size=65536 preallocation='falloc' lazy_refcounts=off 
qemu-img: disk.img: Could not preallocate data for the new file: Bad file descriptor

On a regular (XFS) filesystem:

$ qemu-img create -f qcow2 -o preallocation=falloc disk.img 262144
Formatting 'disk.img', fmt=qcow2 size=262144 encryption=off cluster_size=65536 preallocation='falloc' lazy_refcounts=off

Version-Release number of selected component (if applicable):

qemu-img-1.5.3-101.el7.x86_64

How reproducible:

100%

Steps to Reproduce:
1. Run the 'qemu-img' command above on NFS.

--- Additional comment from Richard W.M. Jones on 2015-09-22 08:30:12 BRT ---

Weirdly the same thing happens with upstream qemu (also compiled on RHEL 7).

$ ./qemu-img create -f qcow2 -o preallocation=falloc disk.img 262144
Formatting 'disk.img', fmt=qcow2 size=262144 encryption=off cluster_size=65536 preallocation=falloc lazy_refcounts=off refcount_bits=16
qemu-img: disk.img: Could not preallocate data for the new file: Bad file descriptor
$ ./qemu-img --version
qemu-img version 2.4.50, Copyright (c) 2004-2008 Fabrice Bellard

--- Additional comment from Richard W.M. Jones on 2015-09-22 08:34:25 BRT ---

Could be a kernel thing I guess?

stat("disk.img", {st_mode=S_IFREG|0644, st_size=524288, ...}) = 0
open("disk.img", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0644) = 8
ftruncate(8, 524288)                    = 0
fallocate(8, 0, 0, 524288)              = -1 EOPNOTSUPP (Operation not supported)
fstat(8, {st_mode=S_IFREG|0644, st_size=524288, ...}) = 0
fstatfs(8, {f_type="NFS_SUPER_MAGIC", f_bsize=1048576, f_blocks=100666, f_bfree=57908, f_bavail=52772, f_files=6553600, f_ffree=5574214, f_fsid={0, 0}, f_namelen=255, f_frsize=1048576}) = 0
pread(8, 0x7fcd2ce9ac80, 1, 524287)     = -1 EBADF (Bad file descriptor)
close(8)                                = 0
write(2, "qemu-img:", 9qemu-img:)                = 9
write(2, " ", 1 )                        = 1
write(2, "disk.img: Could not preallocate "..., 74disk.img: Could not preallocate data for the new file: Bad file descriptor) = 74

--- Additional comment from Richard W.M. Jones on 2015-09-22 09:02 BRT ---

Reproducer.  Simply compile this program and run it on an NFS
mounted filesystem.

$ gcc -Wall bz1265196.c -o bz1265196
$ ./bz1265196 
posix_fallocate failed: Bad file descriptor

On a regular filesystem there is no error.

--- Additional comment from Richard W.M. Jones on 2015-09-22 09:03:02 BRT ---

kernel-3.10.0-306.0.1.el7.x86_64
glibc-2.17-105.el7.x86_64

--- Additional comment from Richard W.M. Jones on 2015-09-22 09:17:20 BRT ---

Also happens in RHEL 7.1:

kernel-3.10.0-229.el7.x86_64
glibc-2.17-78.el7.x86_64

Therefore it's NOT a regression.

--- Additional comment from J. Bruce Fields on 2015-09-29 11:34:52 BRT ---

(In reply to Richard W.M. Jones from comment #3)
> Created attachment 1075791 [details]
> bz1265196.c
> 
> Reproducer.  Simply compile this program and run it on an NFS
> mounted filesystem.
> 
> $ gcc -Wall bz1265196.c -o bz1265196
> $ ./bz1265196 
> posix_fallocate failed: Bad file descriptor
> 
> On a regular filesystem there is no error.

The strace shows that fallocate() is actually returning EOPNOTSUPP, which looks correct to me (the version of NFS we're mounting with does not support fallocate).

Then the strace shows the test attempting a pread, which returns EBADF.  That is also correct, since the file was open O_WRONLY.

So apparently posix_fallocate is returning with the error set to EBADF.  I suspect it should be returning EOPNOTSUPP?  That looks like a bug in the posix_fallocate implementation to me; changing component to glibc in hopes glibc folks can shed some light.

--- Additional comment from Richard W.M. Jones on 2015-09-29 11:55:45 BRT ---

I see now this is also a problem with upstream glibc.

The cause seems to be:
https://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/posix/posix_fallocate64.c;h=fb2dac6e1341938034b7a3ea09d4409d0f26aab1;hb=HEAD#l107

--- Additional comment from J. Bruce Fields on 2015-09-29 12:32:29 BRT ---

Also note that NFS versions since 4.2 have fallocate support.  That fallocate support is available in RHEL7.2 (see https://bugzilla.redhat.com/show_bug.cgi?id=1079385), but NFSv4.2 is not the default, so EOPNOTSUPP will continue to be the default behavior.

--- Additional comment from Carlos O'Donell on 2015-09-29 12:38:23 BRT ---

Please see the manual:
http://www.gnu.org/software/libc/manual/html_node/Storage-Allocation.html#index-posix_005ffallocate64

I have just committed a change to the manual to clarify that O_WRONLY files are not supported either:
https://www.sourceware.org/ml/libc-alpha/2015-09/msg00729.html

Unfortunately this is exactly as desgiend, there is little room to engineer a better solution, particularly for O_WRONLY which won't allow an atomic-CAS/SIGBUS solution for emulation.

You must open the file read/write for posix_fallocate emulation to work on filesystems that have no native fallocate support, or you must use linux fallocate directly to do the work (as described in the manual).

I'm open to other ideas.

Does this answer your questions?

--- Additional comment from J. Bruce Fields on 2015-09-29 13:04:26 BRT ---

(In reply to Carlos O'Donell from comment #9)
> Please see the manual:
> http://www.gnu.org/software/libc/manual/html_node/Storage-Allocation.
> html#index-posix_005ffallocate64
> 
> I have just committed a change to the manual to clarify that O_WRONLY files
> are not supported either:
> https://www.sourceware.org/ml/libc-alpha/2015-09/msg00729.html

That looks helpful, thanks!

Unfortunately I didn't think to check that manual.  At least on Fedora 22 the posix_fallocate man page is missing these details.

> Unfortunately this is exactly as desgiend, there is little room to engineer
> a better solution, particularly for O_WRONLY which won't allow an
> atomic-CAS/SIGBUS solution for emulation.

Understood.

> You must open the file read/write for posix_fallocate emulation to work on
> filesystems that have no native fallocate support, or you must use linux
> fallocate directly to do the work (as described in the manual).
> 
> I'm open to other ideas.
> 
> Does this answer your questions?

Sounds like this should be closed (NOTABUG?).

--- Additional comment from Richard W.M. Jones on 2015-09-29 13:43:27 BRT ---

Just hold on with this bug.  I posted a patch to qemu-devel (not
available on the web page yet).

--- Additional comment from Richard W.M. Jones on 2015-09-30 08:55 BRT ---

Kevin took my patch to qemu which changes the open flag from O_WRONLY
to O_RDWR.  So from the qemu point of view this is fixed.

Since qemu-devel is still broken, I have attached the patch which
was accepted to qemu here.

It'd be kind of nice to have this fix in RHEL since it affects the
'virt-v2v -oa preallocated' option.  So I'm going to change the
component of this bug back to qemu.