Bug 1194753 - Storage tier feature
Summary: Storage tier feature
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: core
Version: mainline
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Dan Lambright
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks: 1206517
TreeView+ depends on / blocked
 
Reported: 2015-02-20 16:52 UTC by Dan Lambright
Modified: 2016-06-16 12:42 UTC (History)
4 users (show)

Fixed In Version: glusterfs-3.8rc2
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-06-16 12:42:14 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)

Description Dan Lambright 2015-02-20 16:52:55 UTC
Patches related to the storage tiering feature.

Comment 1 Anand Avati 2015-02-22 16:22:22 UTC
REVIEW: http://review.gluster.org/9724 (cluster/dht: Add tier translator.) posted (#1) for review on master by Dan Lambright (dlambrig)

Comment 2 Anand Avati 2015-02-25 21:17:12 UTC
REVIEW: http://review.gluster.org/9753 (glusterd: CLI commands to create and manage tiered volumes.) posted (#1) for review on master by Dan Lambright (dlambrig)

Comment 3 Anand Avati 2015-02-25 21:20:55 UTC
REVIEW: http://review.gluster.org/9753 (glusterd: CLI commands to create and manage tiered volumes.) posted (#2) for review on master by Dan Lambright (dlambrig)

Comment 4 Anand Avati 2015-02-26 21:06:18 UTC
REVIEW: http://review.gluster.org/9753 (glusterd: CLI commands to create and manage tiered volumes.) posted (#3) for review on master by Dan Lambright (dlambrig)

Comment 5 Anand Avati 2015-02-26 21:12:04 UTC
REVIEW: http://review.gluster.org/9753 (glusterd: CLI commands to create and manage tiered volumes.) posted (#4) for review on master by Dan Lambright (dlambrig)

Comment 6 Anand Avati 2015-02-26 21:56:15 UTC
REVIEW: http://review.gluster.org/9753 (glusterd: CLI commands to create and manage tiered volumes.) posted (#5) for review on master by Dan Lambright (dlambrig)

Comment 7 Anand Avati 2015-02-27 22:18:06 UTC
REVIEW: http://review.gluster.org/9724 (cluster/dht: Add tier translator.) posted (#2) for review on master by Dan Lambright (dlambrig)

Comment 8 Anand Avati 2015-03-03 22:32:26 UTC
REVIEW: http://review.gluster.org/9724 (cluster/dht: Add tier translator.) posted (#3) for review on master by Dan Lambright (dlambrig)

Comment 9 Anand Avati 2015-03-04 17:45:46 UTC
REVIEW: http://review.gluster.org/9753 (glusterd: CLI commands to create and manage tiered volumes.) posted (#6) for review on master by Dan Lambright (dlambrig)

Comment 10 Anand Avati 2015-03-04 17:57:44 UTC
REVIEW: http://review.gluster.org/9753 (glusterd: CLI commands to create and manage tiered volumes.) posted (#7) for review on master by Dan Lambright (dlambrig)

Comment 11 Anand Avati 2015-03-06 19:06:17 UTC
REVIEW: http://review.gluster.org/9753 (glusterd: CLI commands to create and manage tiered volumes.) posted (#8) for review on master by Dan Lambright (dlambrig)

Comment 12 Anand Avati 2015-03-10 13:16:06 UTC
REVIEW: http://review.gluster.org/9724 (cluster/dht: Add tier translator.) posted (#4) for review on master by Dan Lambright (dlambrig)

Comment 13 Anand Avati 2015-03-10 22:11:21 UTC
REVIEW: http://review.gluster.org/9683 (Adding Libgfdb and ChangeTimeRecorder(CTR) Xlator to GlusterFS) posted (#8) for review on master by Dan Lambright (dlambrig)

Comment 14 Anand Avati 2015-03-14 18:33:08 UTC
REVIEW: http://review.gluster.org/9724 (cluster/dht: Add tier translator.) posted (#5) for review on master by Dan Lambright (dlambrig)

Comment 15 Anand Avati 2015-03-14 20:43:24 UTC
REVIEW: http://review.gluster.org/9724 (cluster/dht: Add tier translator.) posted (#6) for review on master by Dan Lambright (dlambrig)

Comment 16 Anand Avati 2015-03-17 13:19:06 UTC
REVIEW: http://review.gluster.org/9683 (Adding Libgfdb and ChangeTimeRecorder(CTR) Xlator to GlusterFS) posted (#9) for review on master by Joseph Fernandes (josferna)

Comment 17 Anand Avati 2015-03-17 15:01:44 UTC
REVIEW: http://review.gluster.org/9683 (Adding Libgfdb and ChangeTimeRecorder(CTR) Xlator to GlusterFS) posted (#10) for review on master by Joseph Fernandes (josferna)

Comment 18 Anand Avati 2015-03-17 16:13:45 UTC
REVIEW: http://review.gluster.org/9683 (Adding Libgfdb and ChangeTimeRecorder(CTR) Xlator to GlusterFS) posted (#11) for review on master by Joseph Fernandes (josferna)

Comment 19 Anand Avati 2015-03-17 18:58:00 UTC
REVIEW: http://review.gluster.org/9724 (cluster/dht: Add tier translator.) posted (#7) for review on master by Dan Lambright (dlambrig)

Comment 20 Anand Avati 2015-03-17 19:15:51 UTC
REVIEW: http://review.gluster.org/9724 (cluster/dht: Add tier translator.) posted (#8) for review on master by Dan Lambright (dlambrig)

Comment 21 Anand Avati 2015-03-17 19:30:06 UTC
REVIEW: http://review.gluster.org/9753 (glusterd: CLI commands to create and manage tiered volumes.) posted (#9) for review on master by Dan Lambright (dlambrig)

Comment 22 Anand Avati 2015-03-17 20:03:47 UTC
REVIEW: http://review.gluster.org/9683 (Adding Libgfdb and ChangeTimeRecorder(CTR) Xlator to GlusterFS) posted (#12) for review on master by Joseph Fernandes (josferna)

Comment 23 Anand Avati 2015-03-17 21:05:42 UTC
REVIEW: http://review.gluster.org/9683 (Adding Libgfdb and ChangeTimeRecorder(CTR) Xlator to GlusterFS) posted (#13) for review on master by Joseph Fernandes (josferna)

Comment 24 Anand Avati 2015-03-17 22:59:19 UTC
REVIEW: http://review.gluster.org/9724 (cluster/dht: Add tier translator.) posted (#9) for review on master by Dan Lambright (dlambrig)

Comment 25 Anand Avati 2015-03-18 07:18:52 UTC
REVIEW: http://review.gluster.org/9753 (glusterd: CLI commands to create and manage tiered volumes.) posted (#10) for review on master by Dan Lambright (dlambrig)

Comment 26 Anand Avati 2015-03-18 13:43:22 UTC
REVIEW: http://review.gluster.org/9683 (Adding Libgfdb and ChangeTimeRecorder(CTR) Xlator to GlusterFS) posted (#14) for review on master by Joseph Fernandes (josferna)

Comment 27 Anand Avati 2015-03-18 13:44:36 UTC
REVIEW: http://review.gluster.org/9683 (Adding Libgfdb to GlusterFS) posted (#15) for review on master by Joseph Fernandes (josferna)

Comment 28 Anand Avati 2015-03-18 14:28:47 UTC
REVIEW: http://review.gluster.org/9683 (Adding Libgfdb to GlusterFS) posted (#16) for review on master by Joseph Fernandes (josferna)

Comment 29 Anand Avati 2015-03-18 14:28:51 UTC
REVIEW: http://review.gluster.org/9935 (Adding ChangeTimeRecorder(CTR) Xlator to GlusterFS) posted (#1) for review on master by Joseph Fernandes (josferna)

Comment 30 Anand Avati 2015-03-18 15:11:36 UTC
REVIEW: http://review.gluster.org/9683 (Adding Libgfdb to GlusterFS) posted (#17) for review on master by Joseph Fernandes (josferna)

Comment 31 Anand Avati 2015-03-18 15:11:39 UTC
REVIEW: http://review.gluster.org/9935 (Adding ChangeTimeRecorder(CTR) Xlator to GlusterFS) posted (#2) for review on master by Joseph Fernandes (josferna)

Comment 32 Anand Avati 2015-03-18 17:36:46 UTC
COMMIT: http://review.gluster.org/9683 committed in master by Vijay Bellur (vbellur) 
------
commit 87c7fa3cfdadca4ee883daf84373302a42ad5fdc
Author: Joseph Fernandes <josferna>
Date:   Wed Feb 18 19:45:23 2015 +0530

    Adding Libgfdb to GlusterFS
    
    *************************************************************************
    			Libgfdb						|
    *************************************************************************
    Libgfdb provides abstract mechanism to record extra/rich metadata
    required for data maintenance, such as data tiering/classification.
    It provides consumer with API for recording and querying, keeping
    the consumer abstracted from the data store used beneath for storing data.
    It works in a plug-and-play model, where data stores can be plugged-in.
    Presently we have plugin for Sqlite3. In the future will provide recording
    and querying performance optimizer. In the current implementation the schema
    of metadata is fixed.
    
    Schema:
    ~~~~~~
          GF_FILE_TB Table:
          ~~~~~~~~~~~~~~~~~
          This table has one entry per file inode. It holds the metadata required to
          make decisions in data maintenance.
          GF_ID (Primary key)	: File GFID (Universal Unique IDentifier in the namespace)
          W_SEC, W_MSEC 		: Write wind time in sec & micro-sec
          UW_SEC, UW_MSEC		: Write un-wind time in sec & micro-sec
          W_READ_SEC, W_READ_MSEC 	: Read wind time in sec & micro-sec
          UW_READ_SEC, UW_READ_MSEC : Read un-wind time in sec & micro-sec
          WRITE_FREQ_CNTR INTEGER	: Write Frequency Counter
          READ_FREQ_CNTR INTEGER	: Read Frequency Counter
    
          GF_FLINK_TABLE:
          ~~~~~~~~~~~~~~
          This table has all the hardlinks to a file inode.
          GF_ID		: File GFID               (Composite Primary Key)``|
          GF_PID		: Parent Directory GFID  (Composite Primary Key)   |-> Primary Key
          FNAME 		: File Base Name          (Composite Primary Key)__|
          FPATH 		: File Full Path (Its redundant for now, this will go)
          W_DEL_FLAG 	: This Flag is used for crash consistancy, when a link is unlinked.
                      	  i.e Set to 1 during unlink wind and during unwind this record
                              is deleted
          LINK_UPDATE 	: This Flag is used when a link is changed i.e rename.
                              Set to 1 when rename wind and set to 0 in rename unwind
    
    Libgfdb API:
    ~~~~~~~~~~~
    Refer libglusterfs/src/gfdb/gfdb_data_store.h
    
    Change-Id: I2e9fbab3878ce630a7f41221ef61017dc43db11f
    BUG: 1194753
    Signed-off-by: Joseph Fernandes <josferna>
    Signed-off-by: Dan Lambright <dlambrig>
    Signed-off-by: Joseph Fernandes <josferna>
    Reviewed-on: http://review.gluster.org/9683
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Vijay Bellur <vbellur>

Comment 33 Anand Avati 2015-03-19 05:47:03 UTC
REVIEW: http://review.gluster.org/9935 (Adding ChangeTimeRecorder(CTR) Xlator to GlusterFS) posted (#3) for review on master by Vijay Bellur (vbellur)

Comment 34 Anand Avati 2015-03-19 08:31:41 UTC
COMMIT: http://review.gluster.org/9935 committed in master by Vijay Bellur (vbellur) 
------
commit b4a0db5ee3b827268fe1aeeed32ad7d417460c50
Author: Joseph Fernandes <josferna>
Date:   Wed Mar 18 19:55:31 2015 +0530

    Adding ChangeTimeRecorder(CTR) Xlator to GlusterFS
    
    **********************************************************************
                                ChangeTimeRecorder(CTR) Xlator           |
    **********************************************************************
    ChangeTimeRecorder(CTR) is server side xlator(translator) which sits
    just above posix xlator. The main role of this xlator is to record the
    access/write patterns on a file residing the brick. It records the
    read(only data) and write(data and metadata) times and also count on
    how many times a file is read or written. This xlator also captures
    the hard links to a file(as its required by data tiering to move
    files).
    
    CTR Xlator is the consumer of libgfdb.
    
    To Enable/Disable CTR Xlator:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
     gluster volume set <volume-name> features.ctr-enabled {on/off}
    
    To Enable/Disable Frequency Counter Recording in CTR Xlator:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
     gluster volume set <volume-name> features.record-counters {on/off}
    
    Change-Id: I5d3cf056af61ac8e3f8250321a27cb240a214ac2
    BUG: 1194753
    Signed-off-by: Joseph Fernandes <josferna>
    Reviewed-on: http://review.gluster.org/9935
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Vijay Bellur <vbellur>

Comment 35 Anand Avati 2015-03-19 11:05:17 UTC
REVIEW: http://review.gluster.org/9753 (glusterd: CLI commands to create and manage tiered volumes.) posted (#11) for review on master by Dan Lambright (dlambrig)

Comment 36 Anand Avati 2015-03-19 11:41:17 UTC
REVIEW: http://review.gluster.org/9724 (cluster/dht: Add tier translator.) posted (#10) for review on master by Dan Lambright (dlambrig)

Comment 37 Anand Avati 2015-03-19 13:32:34 UTC
COMMIT: http://review.gluster.org/9753 committed in master by Vijay Bellur (vbellur) 
------
commit 6f71bc02df5bd177c2f5dbf4e54b2af1525ab979
Author: Dan Lambright <dlambrig>
Date:   Wed Feb 25 16:11:23 2015 -0500

    glusterd: CLI commands to create and manage tiered volumes.
    
    A tiered volume is a normal volume with some number of new bricks
    representing "hot" storage. The "hot" bricks can be attached or
    detached dynamically to a normal volume. When this happens, a new graph
    is constructed. The root of the new graph is an instance of the tier
    translator. One subvolume of the tier translator leads to the old volume,
    and another leads to the new hot bricks.
    
    attach-tier <VOLNAME> [<replica> <COUNT>] <NEW-BRICK> ... [force]
    
    volume detach-tier <VOLNAME> [replica <COUNT>] <BRICK>
    ... <start|stop|status|commit|force>
    
    gluster volume rebalance <volume> tier start
    
    gluster volume rebalance <volume> tier stop
    
    gluster volume rebalance <volume> tier status
    
    The "tier start" CLI command starts a server side daemon. The daemon
    initiates file level migration based on caching policies. The daemon's
    status can be monitored and stopped.
    
    Note development on the "tier status" command is incomplete. It will be
    added in a subsequent patch.
    
    When the "hot" storage is detached, the tier translator is removed
    from the graph and the tiered volume reverts to its original state as
    described in the volume's info file.
    
    For more background and design see the feature page [1].
    
    [1]
    http://www.gluster.org/community/documentation/index.php/Features/data-classification
    
    Change-Id: Ic8042ce37327b850b9e199236e5be3dae95d2472
    BUG: 1194753
    Signed-off-by: Dan Lambright <dlambrig>
    Reviewed-on: http://review.gluster.org/9753
    Reviewed-by: Vijay Bellur <vbellur>
    Tested-by: Vijay Bellur <vbellur>

Comment 38 Anand Avati 2015-03-19 14:32:22 UTC
REVIEW: http://review.gluster.org/9724 (cluster/dht: Add tier translator.) posted (#11) for review on master by Vijay Bellur (vbellur)

Comment 39 Anand Avati 2015-03-19 15:18:27 UTC
REVIEW: http://review.gluster.org/9724 (cluster/dht: Add tier translator.) posted (#12) for review on master by Dan Lambright (dlambrig)

Comment 40 Anand Avati 2015-03-19 15:50:18 UTC
REVIEW: http://review.gluster.org/9946 (build: pass the correct CFLAGS and LDFLAGS on to libgfdb) posted (#1) for review on master by Niels de Vos (ndevos)

Comment 41 Anand Avati 2015-03-19 16:12:54 UTC
REVIEW: http://review.gluster.org/9946 (build: pass the correct CFLAGS and LDFLAGS when linking (to) libgfdb) posted (#2) for review on master by Niels de Vos (ndevos)

Comment 42 Anand Avati 2015-03-19 18:14:18 UTC
REVIEW: http://review.gluster.org/9724 (cluster/dht: Add tier translator.) posted (#13) for review on master by Dan Lambright (dlambrig)

Comment 43 Anand Avati 2015-03-19 20:16:59 UTC
COMMIT: http://review.gluster.org/9946 committed in master by Kaleb KEITHLEY (kkeithle) 
------
commit e3f2ebfe6b621e84b0386d5a0613829241401d93
Author: Niels de Vos <ndevos>
Date:   Thu Mar 19 16:48:16 2015 +0100

    build: pass the correct CFLAGS and LDFLAGS when linking (to) libgfdb
    
    Change-Id: Id9a7d0f457d9759ab7d0a52a4000b5ae36d211f8
    BUG: 1194753
    Signed-off-by: Niels de Vos <ndevos>
    Reviewed-on: http://review.gluster.org/9946
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Kaleb KEITHLEY <kkeithle>

Comment 44 Anand Avati 2015-03-19 23:21:27 UTC
REVIEW: http://review.gluster.org/9724 (cluster/dht: Add tier translator.) posted (#14) for review on master by Dan Lambright (dlambrig)

Comment 45 Anand Avati 2015-03-20 13:25:06 UTC
REVIEW: http://review.gluster.org/9724 (cluster/dht: Add tier translator.) posted (#15) for review on master by Dan Lambright (dlambrig)

Comment 46 Anand Avati 2015-03-20 14:05:49 UTC
REVIEW: http://review.gluster.org/9957 (tiering/CTR:build CTR only if tiering is enabled) posted (#1) for review on master by mohammed rafi  kc (rkavunga)

Comment 47 Anand Avati 2015-03-20 14:07:59 UTC
REVIEW: http://review.gluster.org/9957 (tiering/CTR:build CTR only if tiering is enabled) posted (#2) for review on master by mohammed rafi  kc (rkavunga)

Comment 48 Niels de Vos 2015-03-20 15:47:32 UTC
*** Bug 1204008 has been marked as a duplicate of this bug. ***

Comment 49 Anand Avati 2015-03-20 16:48:01 UTC
COMMIT: http://review.gluster.org/9957 committed in master by Kaleb KEITHLEY (kkeithle) 
------
commit 24e01e407a04db4983e45fc5338e4a732df7da28
Author: Mohammed Rafi KC <rkavunga>
Date:   Fri Mar 20 19:28:22 2015 +0530

    tiering/CTR:build CTR only if tiering is enabled
    
    Thanks to Niels for this fix
    
    Change-Id: I9a13c3de3ed5d4eb06c6af61a2519bf27f1b6259
    BUG: 1194753
    Signed-off-by: Mohammed Rafi KC <rkavunga>
    Reviewed-on: http://review.gluster.org/9957
    Reviewed-by: Niels de Vos <ndevos>
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Kaleb KEITHLEY <kkeithle>

Comment 50 Anand Avati 2015-03-21 16:50:34 UTC
COMMIT: http://review.gluster.org/9724 committed in master by Vijay Bellur (vbellur) 
------
commit 20355992e8eed7d3ed78a23bc7e922d6ae94860d
Author: Dan Lambright <dlambrig>
Date:   Sun Feb 22 11:05:58 2015 -0500

    cluster/dht: Add tier translator.
    
    The tier translator shares most of DHT's code. It differs in how
    subvolumes are chosen for I/Os, and how file migration (cache promotion
    and demotion) is managed. That different functionality is split to either
    DHT or tier logic according to the "tier_methods" structure.
    
    A cache promotion and demotion thread is created in a manner
    similar to the rebalance daemon. The thread operates a timing
    wheel which periodically checks for promotion and demotion candidates
    (files). Candidates are queued and then migrated. Candidates must exist on
    the same node as the daemon and meet other critera per caching policies.
    
    This patch has two authors (Dan Lambright and Joseph Fernandes). Dan
    did the DHT changes and Joe wrote the cache policies. The fix depends on
    DHT readidr changes and the database library which have been submitted
    separately.  Header files in libglusterfs/src/gfdb should be reviewed in
    patch 9683.
    
    For more background and design see the feature page [1].
    
    [1]
    http://www.gluster.org/community/documentation/index.php/Features/data-classification
    
    Change-Id: Icc26c517ccecf5c42aef039f5b9c6f7afe83e46c
    BUG: 1194753
    Signed-off-by: Dan Lambright <dlambrig>
    Reviewed-on: http://review.gluster.org/9724
    Reviewed-by: Vijay Bellur <vbellur>
    Tested-by: Vijay Bellur <vbellur>

Comment 51 Anand Avati 2015-03-22 12:42:00 UTC
REVIEW: http://review.gluster.org/9964 (build: pass SQLITE_CFLAGS while building dht files that include tier.h) posted (#1) for review on master by Niels de Vos (ndevos)

Comment 52 Anand Avati 2015-03-22 13:50:28 UTC
REVIEW: http://review.gluster.org/9964 (build: pass SQLITE_CFLAGS while building dht files that include tier.h) posted (#2) for review on master by Niels de Vos (ndevos)

Comment 53 Anand Avati 2015-03-22 13:50:37 UTC
REVIEW: http://review.gluster.org/9965 (build: tier.h does not need to include sys/xattr.h) posted (#1) for review on master by Niels de Vos (ndevos)

Comment 54 Anand Avati 2015-03-22 19:30:59 UTC
REVIEW: http://review.gluster.org/9966 (cluster/dht: Fix spurious failure in tier test.) posted (#1) for review on master by Dan Lambright (dlambrig)

Comment 55 Anand Avati 2015-03-22 21:44:57 UTC
REVIEW: http://review.gluster.org/9966 (cluster/dht: Fix spurious failure in tier test.) posted (#2) for review on master by Dan Lambright (dlambrig)

Comment 56 Anand Avati 2015-03-23 09:19:01 UTC
COMMIT: http://review.gluster.org/9966 committed in master by Niels de Vos (ndevos) 
------
commit 507f3251f39405332fc1181e7004b60637f26be6
Author: Dan Lambright <dlambrig>
Date:   Sun Mar 22 15:27:14 2015 -0400

    cluster/dht: Fix spurious failure in tier test.
    
    Need to wait for a few seconds for rebalancing to complete
    before stopping volume.
    
    Change-Id: Ib81c02645240e7d74ebfb3e31ccbc612fc77b119
    BUG: 1194753
    Signed-off-by: Dan Lambright <dlambrig>
    Reviewed-on: http://review.gluster.org/9966
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Niels de Vos <ndevos>

Comment 57 Anand Avati 2015-03-23 09:45:55 UTC
COMMIT: http://review.gluster.org/9964 committed in master by Niels de Vos (ndevos) 
------
commit e47fd239db2da78081daf1f451313956b53435ef
Author: Niels de Vos <ndevos>
Date:   Sun Mar 22 13:38:03 2015 +0100

    build: pass SQLITE_CFLAGS while building dht files that include tier.h
    
    Building on FreeBSD has been broken by http://review.gluster.org/9724
    which introduces the cluster/tiering xlator. The CFLAGS passed to the
    compiler do not include the path where sqlite3.h can be found.
    
    In fact, an attempt was made to pass the flags on, but a later variable
    overwrite these again.
    
    BUG: 1194753
    Change-Id: I1c890fa9a0d82492726306fe6b03bd50ca985e31
    Signed-off-by: Niels de Vos <ndevos>
    Reviewed-on: http://review.gluster.org/9964
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Sachin Pandit <spandit>

Comment 58 Anand Avati 2015-03-23 09:46:24 UTC
COMMIT: http://review.gluster.org/9965 committed in master by Niels de Vos (ndevos) 
------
commit a2d0d9f60a90951b7b5a59ce6c85cea786f3c31d
Author: Niels de Vos <ndevos>
Date:   Sun Mar 22 14:44:08 2015 +0100

    build: tier.h does not need to include sys/xattr.h
    
    FreeBSD does not have sys/xattr.h, including it in tier.h breaks
    building on FreeBSD. There is nothing in tier.h that seems to require
    definitions from the sys/xattr.h header, just remove it.
    
    BUG: 1194753
    Change-Id: If970272a0ce7728e0f18e5ae026880688ac31408
    Signed-off-by: Niels de Vos <ndevos>
    Reviewed-on: http://review.gluster.org/9965
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Sachin Pandit <spandit>

Comment 59 Anand Avati 2015-03-24 09:56:38 UTC
REVIEW: http://review.gluster.org/9983 (rpm: move libgfdb to the glusterfs-server package) posted (#1) for review on master by Niels de Vos (ndevos)

Comment 60 Anand Avati 2015-03-24 14:34:00 UTC
REVIEW: http://review.gluster.org/9983 (rpm: move libgfdb to the glusterfs-server package) posted (#2) for review on master by Niels de Vos (ndevos)

Comment 61 Anand Avati 2015-03-24 18:39:33 UTC
COMMIT: http://review.gluster.org/9983 committed in master by Kaleb KEITHLEY (kkeithle) 
------
commit 6372bea9583fa77b77b48b2389257da2a115e167
Author: Niels de Vos <ndevos>
Date:   Tue Mar 24 10:54:25 2015 +0100

    rpm: move libgfdb to the glusterfs-server package
    
    libgfdb is only used by processes that run on Gluster servers. There is
    no need to have this library (and its sqlite dependency) on any system
    that installs a glusterfs package.
    
    BUG: 1194753
    Change-Id: I6b7fdc70210ef2a847efd596a0c995f0a4cf5d9b
    Signed-off-by: Niels de Vos <ndevos>
    Reviewed-on: http://review.gluster.org/9983
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Humble Devassy Chirammal <humble.devassy>
    Reviewed-by: Lalatendu Mohanty <lmohanty>
    Reviewed-by: Kaleb KEITHLEY <kkeithle>

Comment 62 Anand Avati 2015-03-25 18:29:23 UTC
REVIEW: http://review.gluster.org/10000 (cluster/dht: Fix coverity bug in tiering code) posted (#1) for review on master by Dan Lambright (dlambrig)

Comment 63 Anand Avati 2015-03-25 20:24:10 UTC
REVIEW: http://review.gluster.org/10000 (cluster/dht: Fix coverity bug in tiering code) posted (#2) for review on master by Dan Lambright (dlambrig)

Comment 64 Anand Avati 2015-04-02 00:36:53 UTC
REVIEW: http://review.gluster.org/10000 (cluster/dht: Fix coverity bug in tiering code) posted (#3) for review on master by Dan Lambright (dlambrig)

Comment 65 Anand Avati 2015-04-03 07:43:56 UTC
REVIEW: http://review.gluster.org/10000 (cluster/dht: Fix coverity bug in tiering code) posted (#4) for review on master by Niels de Vos (ndevos)

Comment 66 Anand Avati 2015-04-04 17:38:05 UTC
COMMIT: http://review.gluster.org/10000 committed in master by Niels de Vos (ndevos) 
------
commit 13dbf333c47f0bc0efffddd4b8b83c4031cb7f36
Author: Dan Lambright <dlambrig>
Date:   Wed Mar 25 14:25:33 2015 -0400

    cluster/dht: Fix coverity bug in tiering code
    
    The bug was:
    
    *** CID 1291734:  Error handling issues  (CHECKED_RETURN)
    /xlators/cluster/dht/src/tier.c: 451 in tier_build_migration_qfile()
    
    The fix is to check the return code to the remove library call.
    It is legal to fail, we just log an INFO level message.
    
    Change-Id: I026eb49276b394efa3b8092ee2cc209c470aacb2
    BUG: 1194753
    Signed-off-by: Dan Lambright <dlambrig>
    Reviewed-on: http://review.gluster.org/10000
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Niels de Vos <ndevos>

Comment 67 Anand Avati 2015-04-15 10:34:32 UTC
REVIEW: http://review.gluster.org/10251 (MAINTAINERS: add maintainers for tier feature) posted (#1) for review on master by Dan Lambright (dlambrig)

Comment 68 Anand Avati 2015-04-16 10:00:17 UTC
REVIEW: http://review.gluster.org/10266 (dht/rebalancer: Marking tiering migration fops) posted (#1) for review on master by Joseph Fernandes (josferna)

Comment 69 Anand Avati 2015-04-16 10:11:14 UTC
REVIEW: http://review.gluster.org/10266 (dht/rebalancer: Marking tiering migration fops) posted (#2) for review on master by Joseph Fernandes (josferna)

Comment 70 Anand Avati 2015-04-18 15:07:52 UTC
REVIEW: http://review.gluster.org/10251 (MAINTAINERS: add maintainers for tier feature) posted (#2) for review on master by Dan Lambright (dlambrig)

Comment 71 Anand Avati 2015-04-29 11:05:17 UTC
REVIEW: http://review.gluster.org/10266 (dht/rebalancer: Marking tiering migration fops) posted (#3) for review on master by Joseph Fernandes (josferna)

Comment 72 Anand Avati 2015-05-01 14:45:32 UTC
COMMIT: http://review.gluster.org/10266 committed in master by Kaleb KEITHLEY (kkeithle) 
------
commit 82d78fa3181dff0969ca3b24e2e071ab2b8064d4
Author: Joseph Fernandes <josferna>
Date:   Thu Apr 16 12:30:16 2015 +0530

    dht/rebalancer: Marking tiering migration fops
    
    This is a follow up patch for http://review.gluster.org/#/c/10080
    In the above, the suggested change in
    http://review.gluster.org/#/c/10080/7/xlators/cluster/dht/src/dht-rebalance.c
    doesnot work. The reason it doesnt work is promotion and demotion are done in
    a multithread way. Whenever a promotion or demotion thread is called, the frame
    of the old sync_op thread is not carried with it. As a result the frame->root->pid
    is not set.
    
    Solution:
    When the file is getting migrated, we get a tiering.migration key_value in the
    xattr dict, so that we pass  this dic key-value when we do syncop_setxattr()
    to do  data migration and set the frame->root->pid GF_CLIENT_PID_TIER_DEFRAG
    in dht_setxattr() just before calling dht_start_rebalance_task().
    
    Change-Id: I86fef2d961b32fdd2c0c69d8512cbe846b393404
    BUG: 1194753
    Signed-off-by: Joseph Fernandes <josferna>
    Reviewed-on: http://review.gluster.org/10266
    Reviewed-by: Shyamsundar Ranganathan <srangana>
    Reviewed-by: Susant Palai <spalai>
    Reviewed-by: Dan Lambright <dlambrig>
    Tested-by: Gluster Build System <jenkins.com>

Comment 73 Dan Lambright 2015-05-07 22:32:16 UTC
This bug was used before QE started and is composed of multiple fixes added when development began, henceforth new bugs are created for new problems.

Comment 74 Anand Avati 2015-06-26 15:43:54 UTC
COMMIT: http://review.gluster.org/10251 committed in master by Dan Lambright (dlambrig) 
------
commit add64f005b0afacb6a7c54abf55f1372a6acea52
Author: Dan Lambright <dlambrig>
Date:   Wed Apr 15 06:32:18 2015 -0400

    MAINTAINERS: add maintainers for tier feature
    
    Tiering
    M: Dan Lambright <dlambrig>
    M: Joseph Fernandes <josferna>
    S: Maintained
    F: xlators/cluster/dht/src/tier.c
    F: xlators/features/changetimerecorder
    F: libglusterfs/src/gfdb
    W: http://www.gluster.org/community/documentation/index.php/Features/data-classification
    
    Change-Id: I4b075a287a556551a2b72273901e1d3176f356aa
    BUG: 1194753
    Signed-off-by: Dan Lambright <dlambrig>
    Reviewed-on: http://review.gluster.org/10251

Comment 75 Niels de Vos 2016-06-16 12:42:14 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.0, please open a new bug report.

glusterfs-3.8.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://blog.gluster.org/2016/06/glusterfs-3-8-released/
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user


Note You need to log in before you can comment on or make changes to this bug.