Bug 734339

This is actually an xfs bug.  I ran this test on my box and I got this

btrfs - 21 minutes
xfs - 17 seconds

then I took out the SYNC option and I got

btrfs - 20 seconds
xfs - 17 seconds

turns out xfs has this in its .aio_write


        if (unlikely(file->f_flags & O_DIRECT))
                ret = xfs_file_dio_aio_write(iocb, iovp, nr_segs, pos,
                                                ocount, &iolock);
        else   
                ret = xfs_file_buffered_aio_write(iocb, iovp, nr_segs, pos,
                                                ocount, &iolock);

        xfs_aio_write_isize_update(inode, &iocb->ki_pos, ret);

        if (ret <= 0)     <==== THERE IT IS
                goto out_unlock;

        /* Handle various SYNC-type writes */
        if ((file->f_flags & O_DSYNC) || IS_SYNC(inode)) {
                loff_t end = pos + ret - 1;
                int error;

                xfs_rw_iunlock(ip, iolock);
                error = xfs_file_fsync(file, pos, end,
                                      (file->f_flags & __O_SYNC) ? 0 : 1);
                xfs_rw_ilock(ip, iolock);
                if (error)
                        ret = error;
        }

that should be

if (ret <= 0 && ret != -EIOCBQUEUED)
        goto out_unlock;

But I'm going to cc' Dave on this to get a second opinion.

Adding Christoph to the cc list. This behaviour was changes in upstream XFS in 2.6.33 by commit c355c65 ("xfs: remove IO_ISAIO") which removed this hunk:

@@ -774,9 +772,6 @@ write_retry:
 
        current->backing_dev_info = NULL;
 
-       if (ret == -EIOCBQUEUED && !(ioflags & IO_ISAIO))
-               ret = wait_on_sync_kiocb(iocb);
-
        isize = i_size_read(inode);
        if (unlikely(ret < 0 && ret != -EFAULT && *offset > isize))
                *offset = isize;

Hence Xfs has behaved like this since RHEL6.0.

As it is, I'm not sure what the semantics of "synchronous async IO" are supposed to be (seems rather silly to me), that question needs to be answered as well.

The behaviour post the commit Dave pointed out is defintively incorrect, but just adding the EIOCBQUEUED check like it's done in generic_file_aio_write is not correct either (and thus at least ext4 gets it wrong as well) because we may convert unwritten extents in the I/O completion handler, which this type of O_SYNC I/O won't force out either.

Given that we do not support buffered aio thus we need to do a non-data fsync from the I/O completion handler.  Doing a real log flush from there might be a bit nasty, so I suspect the right answer is to delay the aio complete even further to a callback from the log write I/O completion handler.

(In reply to comment #3)

> As it is, I'm not sure what the semantics of "synchronous async IO" are
> supposed to be (seems rather silly to me), that question needs to be answered
> as well.

Asynchronous I/O simply means we've separated the submission path from the completion path.  It does actually make sense to have O_SYNC async I/O.

Since RHEL 6.2 External Beta has begun, and this bug remains
unresolved, it has been rejected as it is not proposed as
exception or blocker.

Red Hat invites you to ask your support representative to
propose this request, if appropriate and relevant, in the
next release of Red Hat Enterprise Linux.

The tech preview of btrfs in RHEL6 is being ended. Red Hat will not be providing any further updates or security fixes for btrfs in RHEL6, therefore this bug is being closed as "WONTFIX".

The tech preview of btrfs in RHEL7 is continuing, and an update will be provided in RHEL 7.1, in due course. If you are interested in the btrfs tech preview in RHEL7, then please do get in touch to share your use case(s) via the normal channels (e.g. your partner manager, TAM or via a support ticket depending upon which is most approporiate). We are very keen to collect as much information as we can about use cases for btrfs and other future developments in filesystems/storage.