Add the ability to perform encryption in glusterfs
REVIEW: http://review.gluster.org/4667 (Transparent data encryption and metadata authentication) posted (#5) for review on master by Anand Avati (avati)
REVIEW: http://review.gluster.org/4667 (Transparent data encryption and metadata authentication) posted (#6) for review on master by Anand Avati (avati)
REVIEW: http://review.gluster.org/4667 (Transparent data encryption and metadata authentication) posted (#7) for review on master by Anand Avati (avati)
COMMIT: http://review.gluster.org/4667 committed in master by Anand Avati (avati) ------ commit 4efbff29e773a8c59605f87bc3939c9c71b9da16 Author: Edward Shishkin <edward> Date: Wed Mar 13 21:56:46 2013 +0100 Transparent data encryption and metadata authentication .. in the systems with non-trusted server This new functionality can be useful in various cloud technologies. It is implemented via a special encryption/crypt translator,which works on the client side and performs encryption and authentication; 1. Class of supported algorithms The crypt translator can support any atomic symmetric block cipher algorithms (which require to pad plain/cipher text before performing encryption/decryption transform (see glossary in atom.c for definitions). In particular, it can support algorithms with the EOF issue (which require to pad the end of file by extra-data). Crypt translator performs translations user -> (offset, size) -> (aligned-offset, padded-size) ->server (and backward), and resolves individual FOPs (write(), truncate(), etc) to read-modify-write sequences. A volume can contain files encrypted by different algorithms of the mentioned class. To change some option value just reconfigure the volume. Currently only one algorithm is supported: AES_XTS. Example of algorithms, which can not be supported by the crypt translator: 1. Asymmetric block cipher algorithms, which inflate data, e.g. RSA; 2. Symmetric block cipher algorithms with inline MACs for data authentication. 2. Implementation notes. a) Atomic algorithms Since any process in a stackable file system manipulates with local data (which can be obsoleted by local data of another process), any atomic cipher algorithm without proper support can lead to non-POSIX behavior. To resolve the "collisions" we introduce locks: before performing FOP->read(), FOP->write(), etc. the process should first lock the file. b) Algorithms with EOF issue Such algorithms require to pad the end of file with some extra-data. Without proper support this will result in losing information about real file size. Keeping a track of real file size is a responsibility of the crypt translator. A special extended attribute with the name "trusted.glusterfs.crypt.att.size" is used for this purpose. All files contained in bricks of encrypted volume do have "padded" sizes. 3. Non-trusted servers and Metadata authentication We assume that server, where user's data is stored on is non-trusted. It means that the server can be subjected to various attacks directed to reveal user's encrypted personal data. We provide protection against such attacks. Every encrypted file has specific private attributes (cipher algorithm id, atom size, etc), which are packed to a string (so-called "format string") and stored as a special extended attribute with the name "trusted.glusterfs.crypt.att.cfmt". We protect the string from tampering. This protection is mandatory, hardcoded and is always on. Without such protection various attacks (based on extending the scope of per-file secret keys) are possible. Our authentication method has been developed in tight collaboration with Red Hat security team and is implemented as "metadata loader of version 1" (see file metadata.c). This method is NIST-compliant and is based on checking 8-byte per-hardlink MACs created(updated) by FOP->create(), FOP->link(), FOP->unlink(), FOP->rename() by the following unique entities: . file (hardlink) name; . verified file's object id (gfid). Every time, before manipulating with a file, we check it's MACs at FOP->open() time. Some FOPs don't require a file to be opened (e.g. FOP->truncate()). In such cases the crypt translator opens the file mandatory. 4. Generating keys Unique per-file keys are derived by NIST-compliant methods from the a) parent key; b) unique verified object-id of the file (gfid); Per-volume master key, provided by user at mount time is in the root of this "tree of keys". Those keys are used to: 1) encrypt/decrypt file data; 2) encrypt/decrypt file metadata; 3) create per-file and per-link MACs for metadata authentication. 5. Instructions Getting started with crypt translator Example: 1) Create a volume "myvol" and enable encryption: # gluster volume create myvol pepelac:/vols/xvol # gluster volume set myvol encryption on 2) Set location (absolute pathname) of your master key: # gluster volume set myvol encryption.master-key /home/me/mykey 3) Set other options to override default options, if needed. Start the volume. 4) On the client side make sure that the file /home/me/mykey exists and contains proper per-volume master key (that is 256-bit AES key). This key has to be in hex form, i.e. should be represented by 64 symbols from the set {'0', ..., '9', 'a', ..., 'f'}. The key should start at the beginning of the file. All symbols at offsets >= 64 are ignored. 5) Mount the volume "myvol" on the client side: # glusterfs --volfile-server=pepelac --volfile-id=myvol /mnt After successful mount the file which contains master key may be removed. NOTE: Keeping the master key between mount sessions is in user's competence. ********************************************************************** WARNING! Losing the master key will make content of all regular files inaccessible. Mount with improper master key allows to access content of directories: file names are not encrypted. ********************************************************************** 6. Options of crypt translator 1) "master-key": specifies location (absolute pathname) of the file which contains per-volume master key. There is no default location for master key. 2) "data-key-size": specifies size of per-file key for data encryption Possible values: . "256" default value . "512" 3) "block-size": specifies atom size. Possible values: . "512" . "1024" . "2048" . "4096" default value; 7. Test cases Any workload, which involves the following file operations: ->create(); ->open(); ->readv(); ->writev(); ->truncate(); ->ftruncate(); ->link(); ->unlink(); ->rename(); ->readdirp(). 8. TODOs: 1) Currently size of IOs issued by crypt translator is restricted by block_size (4K by default). We can use larger IOs to improve performance. Change-Id: I2601fe95c5c4dc5b22308a53d0cbdc071d5e5cee BUG: 1030058 Signed-off-by: Edward Shishkin <edward> Signed-off-by: Anand Avati <avati> Reviewed-on: http://review.gluster.org/4667 Tested-by: Gluster Build System <jenkins.com>
REVIEW: http://review.gluster.org/6504 (Added test case for crypt translator) posted (#1) for review on master by Edward Shishkin (edward.shishkin)
REVIEW: http://review.gluster.org/6576 (crypt: On calloc failure follow goto statement) posted (#1) for review on master by Harshavardhana (harsha)
COMMIT: http://review.gluster.org/6576 committed in master by Vijay Bellur (vbellur) ------ commit d85726d19432384e2c3dd6ceff4b7b4ec3f8f57a Author: Harshavardhana <harsha> Date: Mon Dec 23 02:52:12 2013 -0800 crypt: On calloc failure follow goto statement At -------------------------------------------- 1423 if (local->vec.iov_base == NULL) { -------------------------------------------- This condition being true leads to NULL pointer to be passed into `memcpy` later at ---------------------------------------------------- 1432 memcpy((char *)local->vec.iov_base + copied, ---------------------------------------------------- Avoid this by clean exit through a goto statement with in the conditional. Change-Id: I2546b7dd634dc251adae8ca39497c4c3ef520f62 BUG: 1030058 Signed-off-by: Harshavardhana <harsha> Reviewed-on: http://review.gluster.org/6576 Reviewed-by: Shyamsundar Ranganathan <srangana> Reviewed-by: Edward Shishkin <edward.shishkin> Tested-by: Edward Shishkin <edward.shishkin> Tested-by: Gluster Build System <jenkins.com>
REVIEW: http://review.gluster.org/6617 (crypt: On calloc failure follow goto statement) posted (#1) for review on release-3.5 by Harshavardhana (harsha)
COMMIT: http://review.gluster.org/6617 committed in release-3.5 by Vijay Bellur (vbellur) ------ commit 2b82cde22d808a70693d8368e1d87f91b30daf24 Author: Harshavardhana <harsha> Date: Mon Dec 23 02:52:12 2013 -0800 crypt: On calloc failure follow goto statement At -------------------------------------------- 1423 if (local->vec.iov_base == NULL) { -------------------------------------------- This condition being true leads to NULL pointer to be passed into `memcpy` later at ---------------------------------------------------- 1432 memcpy((char *)local->vec.iov_base + copied, ---------------------------------------------------- Avoid this by clean exit through a goto statement with in the conditional. Change-Id: I95260767f96107aa96191b18db10908ddda82ee6 BUG: 1030058 Signed-off-by: Harshavardhana <harsha> Reviewed-on: http://review.gluster.org/6617 Reviewed-by: Shyamsundar Ranganathan <srangana> Tested-by: Gluster Build System <jenkins.com>
COMMIT: http://review.gluster.org/6504 committed in master by Anand Avati (avati) ------ commit 2010fb2bc620e6f66ed769b140d92e1d94da1439 Author: Edward Shishkin <edward> Date: Fri Dec 13 21:17:44 2013 +0100 Added test case for crypt translator Change-Id: I7a28a1285c19c3279c2f71e9d9914cf14c761858 BUG: 1030058 Signed-off-by: Edward Shishkin <edward> Reviewed-on: http://review.gluster.org/6504 Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Anand Avati <avati>
REVIEW: http://review.gluster.org/7319 (encryption/crypt: Add mem-accounting for crypt xlator) posted (#1) for review on master by Pranith Kumar Karampuri (pkarampu)
COMMIT: http://review.gluster.org/7319 committed in master by Vijay Bellur (vbellur) ------ commit dc6f7acaeda69ac0765812b3d961197a68ef9bf5 Author: Pranith Kumar K <pkarampu> Date: Sat Mar 22 19:30:45 2014 +0530 encryption/crypt: Add mem-accounting for crypt xlator Without these changes crypt.t crashes when compiled with -DDEBUG Change-Id: I1f7372aa30a09dbe3ae81d1dd598cf36e17fe0b7 BUG: 1030058 Signed-off-by: Pranith Kumar K <pkarampu> Reviewed-on: http://review.gluster.org/7319 Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Vijay Bellur <vbellur>
REVIEW: http://review.gluster.org/8657 (GlusterFS transparent encryption: Improving manageability) posted (#1) for review on master by Edward Shishkin (edward)
REVIEW: http://review.gluster.org/8657 (GlusterFS transparent encryption: Improving manageability) posted (#2) for review on master by Edward Shishkin (edward)
A beta release for GlusterFS 3.6.0 has been released. Please verify if the release solves this bug report for you. In case the glusterfs-3.6.0beta1 release does not have a resolution for this issue, leave a comment in this bug and move the status to ASSIGNED. If this release fixes the problem for you, leave a note and change the status to VERIFIED. Packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update (possibly an "updates-testing" repository) infrastructure for your distribution. [1] http://supercolony.gluster.org/pipermail/gluster-users/2014-September/018836.html [2] http://supercolony.gluster.org/pipermail/gluster-users/
Sorry, glusterfs-3.6.0beta1 still doesn't contain the fixup for this issue [1]. So, changed to ASSIGNED. [1] http://review.gluster.org/8657 Thanks, Edward.
REVIEW: http://review.gluster.org/8657 (GlusterFS transparent encryption: Improving manageability) posted (#3) for review on master by Edward Shishkin (edward)
REVIEW: http://review.gluster.org/8657 (GlusterFS transparent encryption: Improving manageability) posted (#4) for review on master by Edward Shishkin (edward)
Marking as "Tracking", because this bug is used for continuous improvement, not all patches are available in 3.6 or have been merged yet.
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.6.1, please reopen this bug report. glusterfs-3.6.1 has been announced [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://supercolony.gluster.org/pipermail/gluster-users/2014-November/019410.html [2] http://supercolony.gluster.org/mailman/listinfo/gluster-users