Bug 1169517

+++ This bug was initially created as a clone of Bug #1157839 +++

Description of problem:

step #26 of the tests/encryption/crypt.t failed:

How reproducible:

Run tests on NetBsd

Actual results:

=========================
TEST 26 (line 61): diff -u /mnt/glusterfs/0/testfile-symlink
/tmp/patchy-goodfile
diff: /mnt/glusterfs/0/testfile-symlink: Invalid argument

not ok 26 
RESULT 26: 2

Expected results:

TEST 26: OK

Additional info:

1) Complaints in /var/log/glusterfs/mnt-glusterfs-0.log:

[2014-10-23 19:05:40.972909] E [metadata.c:534:open_format_v1] 0-crypt: EMTD verification failed

2) link(2) corrupts meta-data string:

# getfattr -m . -d -e hex /mnt/glusterfs/0/testfile
trusted.glusterfs.crypt.att.cfmt=0x00e0d007795280a78ac90ea90982a00da2ebad6d352e

#ln /mnt/glusterfs/0/testfile /mnt/glusterfs/0/testfile-link

# getfattr -m . -d -e hex /mnt/glusterfs/0/testfile

trusted.glusterfs.crypt.att.cfmt=0x0000000c000480a78ac90ea9098200da2ebad6d352e62ba9283455afd12

--- Additional comment from Anand Avati on 2014-10-27 16:41:59 EDT ---

REVIEW: http://review.gluster.org/8982 (Prevent metadata corruption in the race conditions between FOP->open() and FOP->link().) posted (#1) for review on master by Edward Shishkin (edward)

--- Additional comment from Anand Avati on 2014-10-30 10:07:03 EDT ---

REVIEW: http://review.gluster.org/8982 (Prevent metadata corruption in the race conditions between FOP->open() and FOP->link().) posted (#2) for review on master by Edward Shishkin (edward)

--- Additional comment from Anand Avati on 2014-10-30 15:41:07 EDT ---

REVIEW: http://review.gluster.org/9016 (Build test for https://review.gluster.org/8762) posted (#1) for review on master by Emmanuel Dreyfus (manu)

--- Additional comment from Anand Avati on 2014-10-30 22:39:58 EDT ---

REVIEW: http://review.gluster.org/9016 (Build test for https://review.gluster.org/8762) posted (#2) for review on master by Emmanuel Dreyfus (manu)

--- Additional comment from Anand Avati on 2014-10-30 22:52:48 EDT ---

REVIEW: http://review.gluster.org/9016 (Build test for https://review.gluster.org/8762) posted (#3) for review on master by Emmanuel Dreyfus (manu)

--- Additional comment from Anand Avati on 2014-10-31 00:03:14 EDT ---

REVIEW: http://review.gluster.org/9016 (Build fix: xdrgen) posted (#4) for review on master by Emmanuel Dreyfus (manu)

--- Additional comment from Anand Avati on 2014-11-02 11:41:39 EST ---

COMMIT: http://review.gluster.org/9016 committed in master by Vijay Bellur (vbellur) 
------
commit 12bc39c144aa41a097435f2aab304ddfbbb9b625
Author: Emmanuel Dreyfus <manu>
Date:   Fri Oct 31 04:58:59 2014 +0100

    Build fix: xdrgen
    
    As discovered in https://review.gluster.org/8762, BSD systems fail to
    run xdrgen during glusterfs build. This seems to be caused by a
    difference between BSD make and GNU make whith implcit targets. The
    former seems to use absolute path here, which means we should not
    prepend it with the current directory path, otherwise we have the
    directory path twice and the files cannot be found my make.
    
    BUG: 1157839
    Change-Id: I178123bf6f3d9e963ff5b78839d498f530c05a97
    Signed-off-by: Emmanuel Dreyfus <manu>
    Reviewed-on: http://review.gluster.org/9016
    Reviewed-by: Harshavardhana <harsha>
    Tested-by: Harshavardhana <harsha>
    Reviewed-by: Humble Devassy Chirammal <humble.devassy>

--- Additional comment from Anand Avati on 2014-11-04 06:24:52 EST ---

REVIEW: http://review.gluster.org/9040 (Revert "Build fix: xdrgen") posted (#2) for review on master by Niels de Vos (ndevos)

--- Additional comment from Anand Avati on 2014-11-04 08:41:36 EST ---

COMMIT: http://review.gluster.org/9040 committed in master by Vijay Bellur (vbellur) 
------
commit edeb348a1cfe41aebd700e0689fa1139c1ebe408
Author: Niels de Vos <ndevos>
Date:   Tue Nov 4 03:18:08 2014 -0800

    Revert "Build fix: xdrgen"
    
    This reverts commit 12bc39c144aa41a097435f2aab304ddfbbb9b625
    
    This causes all regression tests to fail. Reverting for now, we can include a
    correct fix later.
    
    Change-Id: I3c8966288f66d0eafa2e94490e3b64a057b4f2c0
    URL: http://supercolony.gluster.org/pipermail/gluster-devel/2014-November/042759.html
    BUG: 1157839
    Reported-by: Emmanuel Dreyfus <manu>
    Signed-off-by: Niels de Vos <ndevos>
    Reviewed-on: http://review.gluster.org/9040
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Vijay Bellur <vbellur>

--- Additional comment from Anand Avati on 2014-11-04 14:56:24 EST ---

REVIEW: http://review.gluster.org/9046 (Build fix: xdrgen) posted (#1) for review on master by Emmanuel Dreyfus (manu)

--- Additional comment from Anand Avati on 2014-11-14 01:16:22 EST ---

COMMIT: http://review.gluster.org/9046 committed in master by Vijay Bellur (vbellur) 
------
commit 77a6917a650a88c12b6c27d37994f8bfef461345
Author: Emmanuel Dreyfus <manu>
Date:   Tue Nov 4 20:53:24 2014 +0100

    Build fix: xdrgen
    
    As discovered in https://review.gluster.org/8762, BSD systems fail to
    run xdrgen during glusterfs build. This seems to be caused by a
    difference between BSD make and GNU make whith implcit targets. The
    former seems to use absolute path here, which means we should not
    prepend it with the current directory path, otherwise we have the
    directory path twice and the files cannot be found my make.
    
    This is a second attempt after I178123bf6f3d9e963ff5b78839d498f530c05a97
    which was broken and reverted in I3c8966288f66d0eafa2e94490e3b64a057b4f2c0
    
    BUG: 1157839
    Change-Id: I797c536c319a156b71a42c82cbaf80bbf17b7234
    Signed-off-by: Emmanuel Dreyfus <manu>
    Reviewed-on: http://review.gluster.org/9046
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Kaleb KEITHLEY <kkeithle>
    Reviewed-by: Vijay Bellur <vbellur>

--- Additional comment from Anand Avati on 2014-12-01 00:41:42 EST ---

COMMIT: http://review.gluster.org/8982 committed in master by Vijay Bellur (vbellur) 
------
commit 45af6c5fc385f2c948bdb387585b3e0a41ccbcde
Author: Edward Shishkin <edward>
Date:   Mon Oct 27 21:35:16 2014 +0100

    Prevent metadata corruption in the race conditions between
    FOP->open() and FOP->link().
    
    Problem: crypt_open() modifies @local->format, which is used
    by crypt_link() to store the updated metadata string on disk.
    This results in metadata corruption.
    
    Fixup: Don't modify @local->format by FOP->open(). Instead
    modify a local copy, allocated in the low-level meta-data
    handler open_format_v1().
    
    Change-Id: I046bb39ddefc33afe59c8d3b1a2fa798298f8499
    BUG: 1157839
    Signed-off-by: Edward Shishkin <edward>
    Reviewed-on: http://review.gluster.org/8982
    Reviewed-by: Emmanuel Dreyfus <manu>
    Tested-by: Emmanuel Dreyfus <manu>
    Tested-by: Gluster Build System <jenkins.com>

--- Additional comment from Anand Avati on 2014-12-01 16:20:17 EST ---

REVIEW: http://review.gluster.org/9225 (Prevent metadata corruption in the race conditions between FOP->open() and FOP->link().) posted (#1) for review on release-3.6 by Edward Shishkin (edward)