Bug 1030058

REVIEW: http://review.gluster.org/4667 (Transparent data encryption and metadata authentication) posted (#5) for review on master by Anand Avati (avati)

REVIEW: http://review.gluster.org/4667 (Transparent data encryption and metadata authentication) posted (#6) for review on master by Anand Avati (avati)

REVIEW: http://review.gluster.org/4667 (Transparent data encryption and metadata authentication) posted (#7) for review on master by Anand Avati (avati)

COMMIT: http://review.gluster.org/4667 committed in master by Anand Avati (avati) 
------
commit 4efbff29e773a8c59605f87bc3939c9c71b9da16
Author: Edward Shishkin <edward>
Date:   Wed Mar 13 21:56:46 2013 +0100

    Transparent data encryption and metadata authentication
    
    .. in the systems with non-trusted server
    
    This new functionality can be useful in various cloud technologies.
    It is implemented via a special encryption/crypt translator,which
    works on the client side and performs encryption and authentication;
    
                  1. Class of supported algorithms
    
    The crypt translator can support any atomic symmetric block cipher
    algorithms (which require to pad plain/cipher text before performing
    encryption/decryption transform (see glossary in atom.c for
    definitions). In particular, it can support algorithms with the EOF
    issue (which require to pad the end of file by extra-data).
    
    Crypt translator performs translations
    user -> (offset, size) -> (aligned-offset, padded-size) ->server
    (and backward), and resolves individual FOPs (write(), truncate(),
    etc) to read-modify-write sequences.
    
    A volume can contain files encrypted by different algorithms of the
    mentioned class. To change some option value just reconfigure the
    volume.
    
    Currently only one algorithm is supported: AES_XTS.
    
    Example of algorithms, which can not be supported by the crypt
    translator:
    
    1. Asymmetric block cipher algorithms, which inflate data, e.g. RSA;
    2. Symmetric block cipher algorithms with inline MACs for data
       authentication.
    
                       2. Implementation notes.
    
    a) Atomic algorithms
    
    Since any process in a stackable file system manipulates with local
    data (which can be obsoleted by local data of another process), any
    atomic cipher algorithm without proper support can lead to non-POSIX
    behavior. To resolve the "collisions" we introduce locks: before
    performing FOP->read(), FOP->write(), etc. the process should first
    lock the file.
    
    b) Algorithms with EOF issue
    
    Such algorithms require to pad the end of file with some extra-data.
    Without proper support this will result in losing information about
    real file size. Keeping a track of real file size is a responsibility
    of the crypt translator. A special extended attribute with the name
    "trusted.glusterfs.crypt.att.size" is used for this purpose. All files
    contained in bricks of encrypted volume do have "padded" sizes.
    
                      3. Non-trusted servers and
                         Metadata authentication
    
    We assume that server, where user's data is stored on is non-trusted.
    It means that the server can be subjected to various attacks directed
    to reveal user's encrypted personal data. We provide protection
    against such attacks.
    
    Every encrypted file has specific private attributes (cipher algorithm
    id, atom size, etc), which are packed to a string (so-called "format
    string") and stored as a special extended attribute with the name
    "trusted.glusterfs.crypt.att.cfmt". We protect the string from
    tampering. This protection is mandatory, hardcoded and is always on.
    Without such protection various attacks (based on extending the scope
    of per-file secret keys) are possible.
    
    Our authentication method has been developed in tight collaboration
    with Red Hat security team and is implemented as "metadata loader of
    version 1" (see file metadata.c). This method is NIST-compliant and is
    based on checking 8-byte per-hardlink MACs created(updated) by
    FOP->create(), FOP->link(), FOP->unlink(), FOP->rename() by the
    following unique entities:
    
    . file (hardlink) name;
    . verified file's object id (gfid).
    
    Every time, before manipulating with a file, we check it's MACs at
    FOP->open() time. Some FOPs don't require a file to be opened (e.g.
    FOP->truncate()). In such cases the crypt translator opens the file
    mandatory.
    
                            4. Generating keys
    
    Unique per-file keys are derived by NIST-compliant methods from the
    
    a) parent key;
    b) unique verified object-id of the file (gfid);
    Per-volume master key, provided by user at mount time is in the root
    of this "tree of keys".
    
    Those keys are used to:
    
    1) encrypt/decrypt file data;
    2) encrypt/decrypt file metadata;
    3) create per-file and per-link MACs for metadata authentication.
    
                              5. Instructions
                     Getting started with crypt translator
    
    Example:
    
    1) Create a volume "myvol" and enable encryption:
    
       # gluster volume create myvol pepelac:/vols/xvol
       # gluster volume set myvol encryption on
    
    2) Set location (absolute pathname) of your master key:
    
       # gluster volume set myvol encryption.master-key /home/me/mykey
    
    3) Set other options to override default options, if needed.
       Start the volume.
    
    4) On the client side make sure that the file /home/me/mykey exists
       and contains proper per-volume master key (that is 256-bit AES
       key). This key has to be in hex form, i.e. should be represented
       by 64 symbols from the set  {'0', ..., '9', 'a', ..., 'f'}.
       The key should start at the beginning of the file. All symbols at
       offsets >= 64 are ignored.
    
    5) Mount the volume "myvol" on the client side:
    
       # glusterfs --volfile-server=pepelac --volfile-id=myvol /mnt
    
       After successful mount the file which contains master key may be
       removed. NOTE: Keeping the master key between mount sessions is in
       user's competence.
    
    **********************************************************************
    
    WARNING! Losing the master key will make content of all regular files
    inaccessible. Mount with improper master key allows to access content
    of directories: file names are not encrypted.
    
    **********************************************************************
    
                   6. Options of crypt translator
    
    1) "master-key": specifies location (absolute pathname) of the file
       which contains per-volume master key. There is no default location
       for master key.
    
    2) "data-key-size": specifies size of per-file key for data encryption
       Possible values:
       . "256" default value
       . "512"
    
    3) "block-size": specifies atom size. Possible values:
       . "512"
       . "1024"
       . "2048"
       . "4096" default value;
    
                           7. Test cases
    
    Any workload, which involves the following file operations:
    
    ->create();
    ->open();
    ->readv();
    ->writev();
    ->truncate();
    ->ftruncate();
    ->link();
    ->unlink();
    ->rename();
    ->readdirp().
    
                            8. TODOs:
    
    1) Currently size of IOs issued by crypt translator is restricted
       by block_size (4K by default). We can use larger IOs to improve
       performance.
    
    Change-Id: I2601fe95c5c4dc5b22308a53d0cbdc071d5e5cee
    BUG: 1030058
    Signed-off-by: Edward Shishkin <edward>
    Signed-off-by: Anand Avati <avati>
    Reviewed-on: http://review.gluster.org/4667
    Tested-by: Gluster Build System <jenkins.com>

REVIEW: http://review.gluster.org/6504 (Added test case for crypt translator) posted (#1) for review on master by Edward Shishkin (edward.shishkin)

REVIEW: http://review.gluster.org/6576 (crypt: On calloc failure follow goto statement) posted (#1) for review on master by Harshavardhana (harsha)

COMMIT: http://review.gluster.org/6576 committed in master by Vijay Bellur (vbellur) 
------
commit d85726d19432384e2c3dd6ceff4b7b4ec3f8f57a
Author: Harshavardhana <harsha>
Date:   Mon Dec 23 02:52:12 2013 -0800

    crypt: On calloc failure follow goto statement
    
    At
    --------------------------------------------
    1423	if (local->vec.iov_base == NULL) {
    --------------------------------------------
    
    This condition being true leads to NULL pointer
    to be passed into `memcpy` later at
    
    ----------------------------------------------------
    1432	memcpy((char *)local->vec.iov_base + copied,
    ----------------------------------------------------
    
    Avoid this by clean exit through a goto statement with
    in the conditional.
    
    Change-Id: I2546b7dd634dc251adae8ca39497c4c3ef520f62
    BUG: 1030058
    Signed-off-by: Harshavardhana <harsha>
    Reviewed-on: http://review.gluster.org/6576
    Reviewed-by: Shyamsundar Ranganathan <srangana>
    Reviewed-by: Edward Shishkin <edward.shishkin>
    Tested-by: Edward Shishkin <edward.shishkin>
    Tested-by: Gluster Build System <jenkins.com>

REVIEW: http://review.gluster.org/6617 (crypt: On calloc failure follow goto statement) posted (#1) for review on release-3.5 by Harshavardhana (harsha)

COMMIT: http://review.gluster.org/6617 committed in release-3.5 by Vijay Bellur (vbellur) 
------
commit 2b82cde22d808a70693d8368e1d87f91b30daf24
Author: Harshavardhana <harsha>
Date:   Mon Dec 23 02:52:12 2013 -0800

    crypt: On calloc failure follow goto statement
    
    At
    --------------------------------------------
    1423	if (local->vec.iov_base == NULL) {
    --------------------------------------------
    
    This condition being true leads to NULL pointer
    to be passed into `memcpy` later at
    
    ----------------------------------------------------
    1432	memcpy((char *)local->vec.iov_base + copied,
    ----------------------------------------------------
    
    Avoid this by clean exit through a goto statement with
    in the conditional.
    
    Change-Id: I95260767f96107aa96191b18db10908ddda82ee6
    BUG: 1030058
    Signed-off-by: Harshavardhana <harsha>
    Reviewed-on: http://review.gluster.org/6617
    Reviewed-by: Shyamsundar Ranganathan <srangana>
    Tested-by: Gluster Build System <jenkins.com>

COMMIT: http://review.gluster.org/6504 committed in master by Anand Avati (avati) 
------
commit 2010fb2bc620e6f66ed769b140d92e1d94da1439
Author: Edward Shishkin <edward>
Date:   Fri Dec 13 21:17:44 2013 +0100

    Added test case for crypt translator
    
    Change-Id: I7a28a1285c19c3279c2f71e9d9914cf14c761858
    BUG: 1030058
    Signed-off-by: Edward Shishkin <edward>
    Reviewed-on: http://review.gluster.org/6504
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Anand Avati <avati>

REVIEW: http://review.gluster.org/7319 (encryption/crypt: Add mem-accounting for crypt xlator) posted (#1) for review on master by Pranith Kumar Karampuri (pkarampu)

COMMIT: http://review.gluster.org/7319 committed in master by Vijay Bellur (vbellur) 
------
commit dc6f7acaeda69ac0765812b3d961197a68ef9bf5
Author: Pranith Kumar K <pkarampu>
Date:   Sat Mar 22 19:30:45 2014 +0530

    encryption/crypt: Add mem-accounting for crypt xlator
    
    Without these changes crypt.t crashes when compiled with -DDEBUG
    
    Change-Id: I1f7372aa30a09dbe3ae81d1dd598cf36e17fe0b7
    BUG: 1030058
    Signed-off-by: Pranith Kumar K <pkarampu>
    Reviewed-on: http://review.gluster.org/7319
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Vijay Bellur <vbellur>

REVIEW: http://review.gluster.org/8657 (GlusterFS transparent encryption: Improving manageability) posted (#1) for review on master by Edward Shishkin (edward)

REVIEW: http://review.gluster.org/8657 (GlusterFS transparent encryption: Improving manageability) posted (#2) for review on master by Edward Shishkin (edward)

A beta release for GlusterFS 3.6.0 has been released. Please verify if the release solves this bug report for you. In case the glusterfs-3.6.0beta1 release does not have a resolution for this issue, leave a comment in this bug and move the status to ASSIGNED. If this release fixes the problem for you, leave a note and change the status to VERIFIED.

Packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update (possibly an "updates-testing" repository) infrastructure for your distribution.

[1] http://supercolony.gluster.org/pipermail/gluster-users/2014-September/018836.html
[2] http://supercolony.gluster.org/pipermail/gluster-users/

Sorry, glusterfs-3.6.0beta1 still doesn't contain the fixup for this issue [1].
So, changed to ASSIGNED.

[1] http://review.gluster.org/8657

Thanks,
Edward.

REVIEW: http://review.gluster.org/8657 (GlusterFS transparent encryption: Improving manageability) posted (#3) for review on master by Edward Shishkin (edward)

REVIEW: http://review.gluster.org/8657 (GlusterFS transparent encryption: Improving manageability) posted (#4) for review on master by Edward Shishkin (edward)

Marking as "Tracking", because this bug is used for continuous improvement, not all patches are available in 3.6 or have been merged yet.

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.6.1, please reopen this bug report.

glusterfs-3.6.1 has been announced [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://supercolony.gluster.org/pipermail/gluster-users/2014-November/019410.html
[2] http://supercolony.gluster.org/mailman/listinfo/gluster-users