+++ This bug was initially created as a clone of Bug #1090488 +++

Description of problem:
-----------------------
O_SYNC writes are barrier-ed with barrier feature.
This works well when the access mechanism is FUSE.
But not in the case of libgfapi access.

Version-Release number of selected component (if applicable):
-------------------------------------------------------------
mainline

How reproducible:
-----------------
Always

Steps to Reproduce:
-------------------
1. Write a C Program to write the file in O_SYNC mode using libgfapi access mechanism

Open the file :
glfs_fd_t fd2 = glfs_creat(fs1, file, O_WRONLY|O_SYNC, 0644);

Write to the file:
written = glfs_write( fd2, buffer, sizeof(buffer), 0 ); 

Actual results:
---------------
O_SYNC Writes are not getting blocked, when barrier is enabled

Expected results:
-----------------
O_SYNC writes should get blocked when barrier is enabled


Additional info:
----------------

Providing the test C Program which uses libgfapi to start O_SYNC writes :
#include "api/glfs.h"
#include <stdlib.h>
#include <stdio.h>
#include <sys/stat.h>
#include <string.h>
#include <errno.h>

void barrier_owrite_test1( glfs_t *fs1, char *file) {
    int fd1;
    char buffer[256];
    glfs_fd_t *fd2;
    int written = 0;

    fd1 = open("/home/sas/smallfile", O_RDONLY);
    if( fd1 < 0 ) {
        printf( "Error:%s", strerror(errno) );
        return;
    }

    fd2 = glfs_open(fs1, file, O_WRONLY|O_SYNC|O_CREAT);
    if( fd2 == NULL ) {
        printf( "Error: %s\n",strerror(errno) );
        close( fd1 );
    }

    while( read(fd1,buffer,sizeof(buffer)) ) {
        written = glfs_write( fd2, buffer, sizeof(buffer), 0 );
        if( written <= 0 ) {
            printf( "Error: %s\n", strerror(errno) );
            break;
        }
    }
    close(fd1);
    glfs_close(fd2);
    fprintf( stdout, "file %s written with osync flag\n",file );
}

int main( int argc, char **argv )
{
    struct stat st = {0};
    int ret = -1;

    if ( argc != 3 ) {
        printf ("Usage: %s <volname> <hostname>\n", argv[0]);
        return -1;
    }

    glfs_t *fs = glfs_new (argv[1]);
    if (!fs) {
        fprintf (stderr, "glfs_new: returned NULL\n");
        return 1;
    }

    ret = glfs_set_volfile_server (fs, "tcp", argv[2], 24007);
    ret = glfs_set_logging (fs, "/tmp/gfapi-err.log", 7);
    ret = glfs_init (fs);

    // Below are the tests corresponding FOPS barriered
    barrier_fsync_test1(fs,argv[3]);

    glfs_fini(fs);
}

Information for compilation:
1. Install glusterfs-api and glusterfs-api-devel package
2. gcc -o <exe> $(pkg-config --libs --cflags glusterfs-api) <gfapi_app.c>

--- Additional comment from Anand Avati on 2014-04-24 08:42:54 EDT ---

REVIEW: http://review.gluster.org/7549 (glusterd-server : barrier O_SYNC write incorrect flag check) posted (#1) for review on master by Atin Mukherjee (amukherj)

--- Additional comment from Anand Avati on 2014-04-28 02:51:55 EDT ---

REVIEW: http://review.gluster.org/7549 (barrier : barrier O_SYNC write incorrect flag check) posted (#2) for review on master by Atin Mukherjee (amukherj)

--- Additional comment from Anand Avati on 2014-04-28 05:25:06 EDT ---

REVIEW: http://review.gluster.org/7549 (barrier : barrier O_SYNC write incorrect flag check) posted (#3) for review on master by Atin Mukherjee (amukherj)

--- Additional comment from Anand Avati on 2014-04-28 05:50:31 EDT ---

REVIEW: http://review.gluster.org/7549 (barrier : barrier O_SYNC write incorrect flag check) posted (#4) for review on master by Atin Mukherjee (amukherj)

--- Additional comment from Anand Avati on 2014-04-30 01:46:51 EDT ---

REVIEW: http://review.gluster.org/7549 (barrier : barrier O_SYNC write incorrect flag check) posted (#5) for review on master by Atin Mukherjee (amukherj)

--- Additional comment from Anand Avati on 2014-05-03 10:28:23 EDT ---

COMMIT: http://review.gluster.org/7549 committed in master by Vijay Bellur (vbellur) 
------
commit 11b0ea3d79445e3773d450132121387b67d7bc9a
Author: Atin Mukherjee <amukherj>
Date:   Wed Apr 23 17:21:41 2014 +0530

    barrier : barrier O_SYNC write incorrect flag check
    
    barrier_writev function was doing the following check to determine whether its a
    O_SYNC write or not:
            if (!(flags & O_SYNC))
    The problem here is this flag is not fd's flag and gfapi write does not copy
    open call fd's flag into write flag because of which O_SYNC writes were not
    getting barriered even if barrier was enabled.
    
    The check has been modified as:
            if (!(fd->flags & (O_SYNC | O_DSYNC)))
    
    Change-Id: I07b23852d150b81c7317100ca6d22d082ad897cd
    BUG: 1090488
    Signed-off-by: Atin Mukherjee <amukherj>
    Reviewed-on: http://review.gluster.org/7549
    Reviewed-by: Varun Shastry <vshastry>
    Reviewed-by: Santosh Pradhan <spradhan>
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Vijay Bellur <vbellur>