Description of problem: The pool utilization percentage provided by ceph df command seems to be wrong. I have a 3 way replicated pool "poolTwo". Initially when there were no objects in the pool the ceph df output was as follows: ------------------------------- output 1: [root@dhcp43-177 ~]# ceph df --cluster ceph GLOBAL: SIZE AVAIL RAW USED %RAW USED 131G 129G 2137M 1.58 POOLS: NAME ID USED %USED MAX AVAIL OBJECTS PoolOne 1 1000M 0.74 32738M 9 PoolTwo 2 0 0 32738M 0 -------------------------------- I added an object of size 500M to the pool PoolTwo, after that the ceph df command output is as follows: ---------------------------------- output 2: [root@dhcp43-177 ~]# ceph df --cluster ceph GLOBAL: SIZE AVAIL RAW USED %RAW USED 131G 128G 3161M 2.34 POOLS: NAME ID USED %USED MAX AVAIL OBJECTS PoolOne 1 1000M 0.74 32226M 9 PoolTwo 2 500M 0.37 32226M 1 ----------------------------------- From the above 2 outputs we can observe that, the USED value has properly changed from 0M to 500M and MAX AVAIL has changed from 32738M to 32226M. Now calculating the %USED: Total size of pool = 32738M (obtained from MAX AVAIL column of output 1, when pool was empty) Current Used = 500M (obtained from USED column of output 2) Percent used = (USED * 100) / TOTAL = (500 * 100) / 32738 = 1.53 But the value shown in %USED column of output 2 is : 0.37 . This is very far from 1.53 Version-Release number of selected component (if applicable): [root@dhcp43-177 ~]# ceph --version ceph version 10.2.2-24.el7cp (ae5487e912fac971db5d84af9ac136054f306f07) How reproducible: Always Steps to Reproduce: 1. Create a ceph pool 2. Fill it with objects and run the "ceph df" command 3. Compare the % utilization from ceph df output and hand calculated % used( formula = (USED * 100) / (USED + MAX AVAIL)). Actual results: There is a difference between the parentage calculated by hand and the one provided in the output. Expected results: The percentage value must be accurate. Additional info: This BZ is raised under the assumption that, percentage utilization = (USED * 100) / (USED + MAX AVAIL) . If this is not the correct way to calculate the percentage, Please update with the right method to calculate it.
Darshan, This is not a bug. It seems like the correct way to calculate this is: USED / GLOBAL SIZE * 100 so in this case you can see that 500 / (131 * 1024) * 100 = .37 I'm not sure what MAX AVAIL represents perhaps Sam could explain that.
In USM we need the pool total size, pool used size and the pool used percentage for all the pools. These details will be shown in web console. We are currently using json format output of ceph df command. It doesn't have the used percentage. So we have to calculate it in usm using the other two parameter that we get from the output. here is a sample output of ceph df command: GLOBAL: SIZE AVAIL RAW USED %RAW USED 33774M 32239M 1534M 4.54 POOLS: NAME ID USED %USED MAX AVAIL OBJECTS PoolOne 1 1000M 2.96 32239M 9 PoolTwo 2 1500M 4.44 32239M 3 same output in JSON format: { "stats": {"total_bytes":35415113728,"total_used_bytes":1609469952,"total_avail_bytes":33805643776}, "pools": [ {"name":"PoolOne","id":1,"stats":{"kb_used":1024001,"bytes_used":1048576228,"max_avail":33805642768,"objects":9}},{"name":"PoolTwo","id":2,"stats":{"kb_used":1536000,"bytes_used":1572864000,"max_avail":33805642768,"objects":3}} ] } So the values for the parameters that USM need are as follows: ------------------------------------------------------------ PoolOne: Pool total size = 35415113728 bytes (obtained from "total_bytes" in above output) Pool used size = 1048576228 bytes (obtained from "bytes_used" under PoolOne in above output) Pool used percentage = (Pool used size * 100) / Pool total size = 2.96 (this exactly matches with value in ceph df output ) PoolTwo: Pool total size = 35415113728 bytes (obtained from "total_bytes" in above output) Pool used size = 1572864000 bytes (obtained from "bytes_used" under PoolTwo in above output) Pool used percentage = (Pool used size * 100) / Pool total size = 4.44 (this exactly matches with value in ceph df output) ------------------------------------------------------------ Can somebody please confirm if above mentioned values for the pool utilization are correct. If no please us know the right values to be considered or if there is any other command that provides pool utilization details more clearly ?
(In reply to Gregory Meno from comment #2) > Darshan, > > This is not a bug. > > It seems like the correct way to calculate this is: > > USED / GLOBAL SIZE * 100 > > so in this case you can see that > 500 / (131 * 1024) * 100 = .37 > > I'm not sure what MAX AVAIL represents perhaps Sam could explain that. Thanks Gregory for pointing that out. Could you please provide your inputs about comment 3.
(In reply to Gregory Meno from comment #2) > Darshan, > > This is not a bug. > > It seems like the correct way to calculate this is: > > USED / GLOBAL SIZE * 100 > > so in this case you can see that > 500 / (131 * 1024) * 100 = .37 > > I'm not sure what MAX AVAIL represents perhaps Sam could explain that. If I have only one storage profile used to create pools then, above calculation matches exactly. But if there are multiple storage profiles used to create different pools, then I feel there is something else to be considered. Because USED / GLOBAL SIZE * 100 doesn't match with the percent. Eg: GLOBAL: SIZE AVAIL RAW USED %RAW USED 67462M 61956M 5506M 8.16 POOLS: NAME ID USED %USED MAX AVAIL OBJECTS df1 1 500M 2.22 15401M 1 er1 2 500M 1.11 30802M 1 pool0 3 500M 2.22 15401M 1 ecpool0 4 500M 1.11 30802M 1 sas-rep 5 500M 0.74 11188M 1 Here first 4 pools are from one storage profile and last pool is from different storage profile. Could you please let us know what is to be considered in above mentioned case.
As Darshan mentioned, we have two hierarchies here - sas and default ceph osd tree ID WEIGHT TYPE NAME UP/DOWN REWEIGHT PRIMARY-AFFINITY -5 0.01199 root sas -4 0.01199 host dhcp42-2-sas 0 0.00600 osd.0 up 1.00000 1.00000 1 0.00600 osd.1 up 1.00000 1.00000 -1 0.05328 root default -2 0.01172 host dhcp42-2 9 0.00586 osd.9 up 1.00000 1.00000 10 0.00586 osd.10 up 1.00000 1.00000 -3 0.01199 host dhcp43-205 2 0.00600 osd.2 up 1.00000 1.00000 3 0.00600 osd.3 up 1.00000 1.00000 -6 0.01785 host dhcp42-79 4 0.00600 osd.4 up 1.00000 1.00000 5 0.00600 osd.5 up 1.00000 1.00000 8 0.00586 osd.8 up 1.00000 1.00000 -7 0.01172 host dhcp43-241 6 0.00586 osd.6 up 1.00000 1.00000 7 0.00586 osd.7 up 1.00000 1.00000 So total available to these two hierarchies would be different, so I don't think we can really depend on the global size to calculate the percentage. Please provide your inputs here
It seems that %USED and MAX AVAIL are calculated by looking at the actual disk usage reported by each osd in the tree used by that pool. USED reflects the nominal amount stored by the pool without accounting for overhead or replication (1000 MB of objects is 1000 MB of USED even if the actual on disk usage is much higher due to replication and overhead). In short, you cannot calculate %USED from MAX_AVAIL and USED.
%USED doesn't show how much of the space for the pool is used, but how much of the space of cluster is used in the pool. With that said if I have several hierarchies the number is not correct. E.g. All OSDs have 10GB os space # ceph osd tree ID WEIGHT TYPE NAME UP/DOWN REWEIGHT PRIMARY-AFFINITY -11 0.00999 root test -10 0.00999 host dhcp-126-101-test 0 0.00999 osd.0 up 1.00000 1.00000 -9 0.02998 root ectest -6 0.00999 host dhcp-126-105-ectest 6 0.00999 osd.6 up 1.00000 1.00000 -7 0.00999 host dhcp-126-102-ectest 2 0.00999 osd.2 up 1.00000 1.00000 -8 0.00999 host dhcp-126-103-ectest 4 0.00999 osd.4 up 1.00000 1.00000 -1 0.03998 root default -2 0.00999 host dhcp-126-101 1 0.00999 osd.1 up 1.00000 1.00000 -3 0.00999 host dhcp-126-102 3 0.00999 osd.3 up 1.00000 1.00000 -4 0.00999 host dhcp-126-103 5 0.00999 osd.5 up 1.00000 1.00000 -5 0.00999 host dhcp-126-105 7 0.00999 osd.7 up 1.00000 1.00000 One pool with replication 4 is created on default hierarchy, so it has 4 OSDs available to use. Logically pool total size is the same as size of one of OSDs, hence total size of the pool is 10GB. I created 7 objects of 1GB size each. Cep df shows: # ceph df GLOBAL: SIZE AVAIL RAW USED %RAW USED 81831M 52877M 28954M 35.38 POOLS: NAME ID USED %USED MAX AVAIL OBJECTS testpool 1 7168M 35.04 3024M 7 As you can see %USED for the pool is the same (or almost the same) as %RAW USED for the cluster.
I'm not sure what you mean by "space for the pool". It's supposed to indicate how much of the space available to the pool is used -- whether by that pool or by the others. It does seem like the %USED here should be higher. How did you create the objects?
Also, can you include the output of ceph osd dump and ceph pg dump?
I'm having a hard time understanding how this blocks the release. Even if it is wrong. Would you please explain it to me what workflow this prevents?
USM is monitoring utilization of each pool and alerts if the thresholds are breached. Also it displays the utilization on the pool listing page. The concern here is that we are displaying wrong information(If it is wrong) and send alerts for wrong data.
Here you can see that osds are almost full: $ ceph osd df --cluster cl1 ID WEIGHT REWEIGHT SIZE USE AVAIL %USE VAR PGS 1 0.00999 1.00000 10228M 9734M 494M 95.16 1.03 128 0 0.00999 1.00000 10228M 9734M 494M 95.16 1.03 128 2 0.00999 1.00000 10228M 9254M 974M 90.47 0.97 128 3 0.00999 1.00000 10228M 9254M 974M 90.47 0.97 128 TOTAL 40915M 37976M 2939M 92.82 MIN/MAX VAR: 0.97/1.03 STDDEV: 2.35 here you can see that pools are only half full: $ ceph --cluster cl1 df GLOBAL: SIZE AVAIL RAW USED %RAW USED 40915M 2939M 37976M 92.82 POOLS: NAME ID USED %USED MAX AVAIL OBJECTS pool1 1 9216M 45.05 974M 9 pool2 2 9696M 47.39 494M 10 I think that root of this problem are crush rules(storage profile concept) made by Console. In this case osd.0 and osd.1 are in "test_profile" Storage profile where is only one pool "pool2". Similar "pool1" contains osd.2 and osd.3 in Storage profile "default". $ ceph --cluster cl1 osd crush dump { "devices": [ { "id": 0, "name": "osd.0" }, { "id": 1, "name": "osd.1" }, { "id": 2, "name": "osd.2" }, { "id": 3, "name": "osd.3" } ], "types": [ { "type_id": 0, "name": "osd" }, { "type_id": 1, "name": "host" }, { "type_id": 2, "name": "chassis" }, { "type_id": 3, "name": "rack" }, { "type_id": 4, "name": "row" }, { "type_id": 5, "name": "pdu" }, { "type_id": 6, "name": "pod" }, { "type_id": 7, "name": "room" }, { "type_id": 8, "name": "datacenter" }, { "type_id": 9, "name": "region" }, { "type_id": 10, "name": "root" } ], "buckets": [ { "id": -1, "name": "default", "type_id": 10, "type_name": "root", "weight": 1310, "alg": "straw", "hash": "rjenkins1", "items": [ { "id": -2, "weight": 0, "pos": 0 }, { "id": -3, "weight": 0, "pos": 1 }, { "id": -4, "weight": 655, "pos": 2 }, { "id": -5, "weight": 655, "pos": 3 } ] }, { "id": -2, "name": "dhcp-126-35", "type_id": 1, "type_name": "host", "weight": 0, "alg": "straw", "hash": "rjenkins1", "items": [] }, { "id": -3, "name": "dhcp-126-67", "type_id": 1, "type_name": "host", "weight": 0, "alg": "straw", "hash": "rjenkins1", "items": [] }, { "id": -4, "name": "dhcp-126-69", "type_id": 1, "type_name": "host", "weight": 655, "alg": "straw", "hash": "rjenkins1", "items": [ { "id": 2, "weight": 655, "pos": 0 } ] }, { "id": -5, "name": "dhcp-126-94", "type_id": 1, "type_name": "host", "weight": 655, "alg": "straw", "hash": "rjenkins1", "items": [ { "id": 3, "weight": 655, "pos": 0 } ] }, { "id": -6, "name": "dhcp-126-67-test_profile", "type_id": 1, "type_name": "host", "weight": 655, "alg": "straw", "hash": "rjenkins1", "items": [ { "id": 1, "weight": 655, "pos": 0 } ] }, { "id": -7, "name": "dhcp-126-35-test_profile", "type_id": 1, "type_name": "host", "weight": 655, "alg": "straw", "hash": "rjenkins1", "items": [ { "id": 0, "weight": 655, "pos": 0 } ] }, { "id": -8, "name": "test_profile", "type_id": 10, "type_name": "root", "weight": 1310, "alg": "straw", "hash": "rjenkins1", "items": [ { "id": -6, "weight": 655, "pos": 0 }, { "id": -7, "weight": 655, "pos": 1 } ] } ], "rules": [ { "rule_id": 0, "rule_name": "replicated_ruleset", "ruleset": 0, "type": 1, "min_size": 1, "max_size": 10, "steps": [ { "op": "take", "item": -1, "item_name": "default" }, { "op": "chooseleaf_firstn", "num": 0, "type": "host" }, { "op": "emit" } ] }, { "rule_id": 1, "rule_name": "test_profile", "ruleset": 1, "type": 1, "min_size": 1, "max_size": 10, "steps": [ { "op": "take", "item": -8, "item_name": "test_profile" }, { "op": "chooseleaf_firstn", "num": 0, "type": "osd" }, { "op": "emit" } ] } ], "tunables": { "choose_local_tries": 0, "choose_local_fallback_tries": 0, "choose_total_tries": 50, "chooseleaf_descend_once": 1, "chooseleaf_vary_r": 1, "chooseleaf_stable": 0, "straw_calc_version": 1, "allowed_bucket_algs": 22, "profile": "firefly", "optimal_tunables": 0, "legacy_tunables": 0, "minimum_required_version": "firefly", "require_feature_tunables": 1, "require_feature_tunables2": 1, "has_v2_rules": 0, "require_feature_tunables3": 1, "has_v3_rules": 0, "has_v4_buckets": 0, "require_feature_tunables5": 0, "has_v5_rules": 0 } } $ ceph --cluster cl1 osd dump epoch 54 fsid e6555ee0-0500-4cf6-b3b9-f28d1b4c95e5 created 2016-08-01 15:00:49.718944 modified 2016-08-02 13:07:27.173811 flags full,sortbitwise pool 1 'pool1' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 128 pgp_num 128 last_change 37 flags hashpspool stripe_width 0 pool 2 'pool2' replicated size 2 min_size 1 crush_ruleset 1 object_hash rjenkins pg_num 128 pgp_num 128 last_change 45 flags hashpspool stripe_width 0 max_osd 4 osd.0 up in weight 1 up_from 4 up_thru 43 down_at 0 last_clean_interval [0,0) 10.34.126.35:6800/11731 10.34.126.35:6801/11731 10.34.126.35:6802/11731 10.34.126.35:6803/11731 exists,up ebe36d64-41a1-4871-912b-2d017592b02b osd.1 up in weight 1 up_from 6 up_thru 43 down_at 0 last_clean_interval [0,0) 10.34.126.67:6800/11903 10.34.126.67:6801/11903 10.34.126.67:6802/11903 10.34.126.67:6803/11903 exists,up 60f8c533-a6fb-47f3-89a5-eec3aeba1996 osd.2 up in weight 1 up_from 8 up_thru 43 down_at 0 last_clean_interval [0,0) 10.34.126.69:6800/11859 10.34.126.69:6801/11859 10.34.126.69:6802/11859 10.34.126.69:6803/11859 exists,up a48e4844-ade2-4090-9441-ba5d3fbb35aa osd.3 up in weight 1 up_from 10 up_thru 43 down_at 0 last_clean_interval [0,0) 10.34.126.94:6800/12083 10.34.126.94:6801/12083 10.34.126.94:6802/12083 10.34.126.94:6803/12083 exists,up 9dad3cb9-c634-4737-8704-a748c5d2f3a7 $ ceph --cluster cl1 pg dump dumped all in format plain version 761 stamp 2016-08-02 15:33:01.024955 last_osdmap_epoch 54 last_pg_scan 54 full_ratio 0.95 nearfull_ratio 0.85 pg_stat objects mip degr misp unf bytes log disklog state state_stamp v reported up up_primary acting acting_primary last_scrub scrub_stamp last_deep_scrub deep_scrub_stamp 2.5c 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:36.559749 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:31:36.559677 0'0 2016-08-02 15:31:36.559677 1.5f 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:16.699599 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:22:16.699500 0'0 2016-08-01 15:19:38.510182 2.5d 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:40.560495 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:31:40.560438 0'0 2016-08-01 15:29:08.246598 1.5e 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:17.038163 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:22:17.038096 0'0 2016-08-01 15:19:38.510167 2.5e 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:03.606755 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:32:03.606698 0'0 2016-08-02 15:32:03.606698 1.5d 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:12.699209 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:22:12.699098 0'0 2016-08-01 15:19:38.510142 2.5f 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:41.561078 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:31:41.560997 0'0 2016-08-01 15:29:08.246630 1.5c 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:07.699231 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:22:07.699156 0'0 2016-08-01 15:19:38.510128 2.58 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:02.606389 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:32:02.606318 0'0 2016-08-01 15:29:08.246470 1.5b 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:16.037816 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:22:16.037696 0'0 2016-08-02 15:22:16.037696 2.59 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:27.558526 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:31:27.558469 0'0 2016-08-01 15:29:08.246506 1.5a 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:06.697997 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:22:06.697825 0'0 2016-08-01 15:19:38.510100 2.5a 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:28.558575 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:31:28.558525 0'0 2016-08-01 15:29:08.246524 1.59 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:03.698309 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:22:03.698221 0'0 2016-08-01 15:19:38.510086 2.5b 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:30.558920 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:31:30.558854 0'0 2016-08-02 15:31:30.558854 1.58 1 0 0 0 0 1073741824 256 256 active+clean 2016-08-02 15:22:02.932075 47'256 54:265 [2,3] 2 [2,3] 2 47'256 2016-08-02 15:22:02.931872 47'256 2016-08-02 15:22:02.931872 2.54 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:58.552821 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:30:58.552756 0'0 2016-08-01 15:29:08.246402 1.57 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:15.037526 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:22:15.037451 0'0 2016-08-01 15:19:38.510045 2.55 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:23.557276 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:31:23.557175 0'0 2016-08-01 15:29:08.246420 1.56 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:12.037179 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:22:12.037046 0'0 2016-08-01 15:19:38.510031 2.56 1 0 0 0 0 1073741824 256 256 active+clean 2016-08-02 15:31:24.558424 47'256 54:251 [1,0] 1 [1,0] 1 47'256 2016-08-02 15:31:24.558366 0'0 2016-08-01 15:29:08.246436 1.55 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:21:41.690603 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:21:41.690509 0'0 2016-08-01 15:19:38.510017 2.57 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:26.557742 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:31:26.557640 0'0 2016-08-02 15:31:26.557640 1.54 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:21:39.690294 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:21:39.690244 0'0 2016-08-02 15:21:39.690244 2.50 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:58.604932 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:31:58.604861 0'0 2016-08-01 15:29:08.246340 1.53 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:21:38.689976 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:21:38.689886 0'0 2016-08-01 15:19:38.509989 2.51 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:59.605035 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:31:59.604971 0'0 2016-08-02 15:31:59.604971 1.52 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:21:36.689740 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:21:36.689685 0'0 2016-08-01 15:19:38.509974 2.52 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:53.551748 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:30:53.551700 0'0 2016-08-02 15:30:53.551700 1.51 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:09.036871 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:22:09.036801 0'0 2016-08-01 15:19:38.509960 2.53 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:56.552280 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:30:56.552176 0'0 2016-08-01 15:29:08.246386 1.50 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:06.036390 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:22:06.036302 0'0 2016-08-01 15:19:38.509945 2.4c 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:53.603720 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:31:53.603677 0'0 2016-08-01 15:29:08.246279 1.4f 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:21:44.030541 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:21:44.030483 0'0 2016-08-01 15:19:38.509931 2.4d 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:49.550234 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:30:49.550175 0'0 2016-08-01 15:29:08.246293 1.4e 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:21:38.029771 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:21:38.029687 0'0 2016-08-01 15:19:38.509916 2.4e 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:57.604662 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:31:57.604524 0'0 2016-08-01 15:29:08.246310 1.4d 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:21:28.688022 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:21:28.687906 0'0 2016-08-01 15:19:38.509902 2.4f 1 0 0 0 0 1073741824 256 256 active+clean 2016-08-02 15:30:50.552003 47'256 54:260 [1,0] 1 [1,0] 1 47'256 2016-08-02 15:30:50.551895 0'0 2016-08-01 15:29:08.246324 1.4c 1 0 0 0 0 1073741824 256 256 active+clean 2016-08-02 15:21:37.030280 47'256 54:243 [3,2] 3 [3,2] 3 47'256 2016-08-02 15:21:37.030189 0'0 2016-08-01 15:19:38.509886 2.48 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:39.548639 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:30:39.548570 0'0 2016-08-02 15:30:39.548570 1.4b 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:21:26.687613 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:21:26.687570 0'0 2016-08-01 15:19:38.509872 2.49 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:40.548786 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:30:40.548678 0'0 2016-08-01 15:29:08.246234 1.4a 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:21:16.686015 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:21:16.685902 0'0 2016-08-01 15:19:38.509850 2.4a 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:52.603512 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:31:52.603453 0'0 2016-08-02 15:31:52.603453 1.49 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:21:14.685280 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:21:14.685217 0'0 2016-08-02 15:21:14.685217 2.4b 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:48.550567 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:30:48.550483 0'0 2016-08-01 15:29:08.246264 1.48 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:21:09.683549 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:21:09.683429 0'0 2016-08-01 15:19:38.509822 1.47 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:21:07.683233 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:21:07.683152 0'0 2016-08-01 15:19:38.509808 2.44 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:20.542972 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:30:20.542859 0'0 2016-08-01 15:29:08.246152 1.46 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:21:35.029395 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:21:35.029329 0'0 2016-08-01 15:19:38.509785 2.45 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:51.603486 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:31:51.603426 0'0 2016-08-01 15:29:08.246167 2.46 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:21.545669 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:30:21.545589 0'0 2016-08-01 15:29:08.246181 1.45 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:21:06.683194 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:21:06.683136 0'0 2016-08-01 15:19:38.509771 2.47 1 0 0 0 0 1073741824 256 256 active+clean 2016-08-02 15:30:38.808751 47'256 54:259 [1,0] 1 [1,0] 1 47'256 2016-08-02 15:30:38.808494 47'256 2016-08-02 15:30:38.808494 1.44 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:21:27.027580 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:21:27.027520 0'0 2016-08-01 15:19:38.509757 1.43 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:21:21.027422 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:21:21.027370 0'0 2016-08-02 15:21:21.027370 2.40 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:44.602360 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:31:44.602294 0'0 2016-08-01 15:29:08.246095 1.42 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:21:03.681943 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:21:03.681869 0'0 2016-08-01 15:19:38.509728 2.41 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:15.541842 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:30:15.541774 0'0 2016-08-01 15:29:08.246109 1.41 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:21:19.026632 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:21:19.026579 0'0 2016-08-01 15:19:38.509714 2.42 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:50.602985 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:31:50.602915 0'0 2016-08-01 15:29:08.246123 1.40 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:21:17.026019 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:21:17.025961 0'0 2016-08-01 15:19:38.509700 2.43 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:16.542387 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:30:16.542317 0'0 2016-08-01 15:29:08.246138 1.3f 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:21:02.681782 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:21:02.681734 0'0 2016-08-01 15:19:38.509686 2.3c 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:40.600900 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:31:40.600820 0'0 2016-08-01 15:29:08.246022 1.3e 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:21:15.025519 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:21:15.025441 0'0 2016-08-01 15:19:38.509671 2.3d 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:12.540675 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:30:12.540600 0'0 2016-08-01 15:29:08.246037 1.3d 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:58.680986 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:20:58.680895 0'0 2016-08-01 15:19:38.509657 2.3e 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:41.601063 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:31:41.600986 0'0 2016-08-02 15:31:41.600986 1.3c 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:53.679762 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:20:53.679720 0'0 2016-08-02 15:20:53.679720 2.3f 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:13.541185 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:30:13.541126 0'0 2016-08-01 15:29:08.246080 1.3b 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:52.679218 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:20:52.679147 0'0 2016-08-02 15:20:52.679147 2.38 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:28.599676 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:31:28.599595 0'0 2016-08-01 15:29:08.245964 1.3a 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:51.678957 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:20:51.678883 0'0 2016-08-01 15:19:38.509615 2.39 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:30.599647 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:31:30.599556 0'0 2016-08-01 15:29:08.245979 1.39 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:21:13.025421 0'0 54:15 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:21:13.025341 0'0 2016-08-01 15:19:38.509601 2.3a 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:34.601014 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:31:34.600942 0'0 2016-08-01 15:29:08.245993 1.38 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:50.678980 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:20:50.678898 0'0 2016-08-01 15:19:38.509586 2.3b 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:35.600976 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:31:35.600915 0'0 2016-08-01 15:29:08.246007 1.37 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:47.678354 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:20:47.678255 0'0 2016-08-01 15:19:38.509571 2.34 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:02.539261 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:30:02.539160 0'0 2016-08-01 15:29:08.245902 1.36 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:45.677999 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:20:45.677905 0'0 2016-08-01 15:19:38.509557 2.35 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:24.597683 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:31:24.597596 0'0 2016-08-01 15:29:08.245920 1.35 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:21:08.024057 0'0 54:15 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:21:08.024009 0'0 2016-08-01 15:19:38.509543 2.36 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:10.540195 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:30:10.540116 0'0 2016-08-02 15:30:10.540116 1.34 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:21:07.023793 0'0 54:15 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:21:07.023711 0'0 2016-08-01 15:19:38.509529 2.37 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:25.598227 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:31:25.598141 0'0 2016-08-01 15:29:08.245950 1.33 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:44.677035 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:20:44.676978 0'0 2016-08-01 15:19:38.509515 2.30 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:19.601361 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:31:19.601265 0'0 2016-08-01 15:29:08.245826 1.32 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:21:06.023550 0'0 54:15 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:21:06.023461 0'0 2016-08-01 15:19:38.509500 2.31 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:58.538210 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:29:58.538160 0'0 2016-08-01 15:29:08.245841 1.31 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:21:04.025322 0'0 54:15 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:21:04.025266 0'0 2016-08-01 15:19:38.509486 2.32 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:21.598014 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:31:21.597942 0'0 2016-08-01 15:29:08.245855 1.30 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:42.676675 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:20:42.676547 0'0 2016-08-01 15:19:38.509472 2.33 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:59.538394 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:29:59.538308 0'0 2016-08-01 15:29:08.245869 1.2f 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:37.675628 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:20:37.675551 0'0 2016-08-01 15:19:38.509447 2.2c 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:55.537498 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:29:55.537424 0'0 2016-08-01 15:29:08.245765 1.2e 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:57.022384 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:20:57.022313 0'0 2016-08-01 15:19:38.509433 2.2d 1 0 0 0 0 1073741824 256 256 active+clean 2016-08-02 15:29:57.537931 51'256 54:246 [1,0] 1 [1,0] 1 51'256 2016-08-02 15:29:57.537875 0'0 2016-08-01 15:29:08.245781 1.2d 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:55.021488 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:20:55.021418 0'0 2016-08-01 15:19:38.509418 2.2e 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:58.596054 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:30:58.595994 0'0 2016-08-01 15:29:08.245797 1.2c 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:35.675957 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:20:35.675769 0'0 2016-08-01 15:19:38.509404 2.2f 1 0 0 0 0 1073741824 256 256 active+clean 2016-08-02 15:31:18.416845 47'256 54:243 [0,1] 0 [0,1] 0 47'256 2016-08-02 15:31:18.416653 47'256 2016-08-02 15:31:18.416653 1.2b 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:33.673738 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:20:33.673672 0'0 2016-08-01 15:19:38.509389 2.28 1 0 0 0 0 503316480 120 120 active+clean 2016-08-02 15:29:48.535736 53'120 54:122 [1,0] 1 [1,0] 1 53'120 2016-08-02 15:29:48.535644 0'0 2016-08-01 15:29:08.245700 1.2a 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:32.673724 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:20:32.673658 0'0 2016-08-01 15:19:38.509374 2.29 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:52.536730 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:29:52.536650 0'0 2016-08-01 15:29:08.245717 1.29 1 0 0 0 0 1073741824 256 256 active+clean 2016-08-02 15:20:30.674322 47'256 54:264 [2,3] 2 [2,3] 2 47'256 2016-08-02 15:20:30.674260 0'0 2016-08-01 15:19:38.509360 2.2a 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:56.592762 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:30:56.592671 0'0 2016-08-01 15:29:08.245732 1.28 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:54.021318 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:20:54.021237 0'0 2016-08-01 15:19:38.509346 2.2b 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:54.537326 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:29:54.537266 0'0 2016-08-01 15:29:08.245747 1.27 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:52.020631 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:20:52.020572 0'0 2016-08-01 15:19:38.509332 2.24 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:44.534294 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:29:44.534223 0'0 2016-08-01 15:29:08.245637 1.26 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:50.020268 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:20:50.020193 0'0 2016-08-01 15:19:38.509318 2.25 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:51.591810 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:30:51.591714 0'0 2016-08-01 15:29:08.245655 1.25 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:49.023211 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:20:49.023127 0'0 2016-08-01 15:19:38.509304 2.26 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:54.592980 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:30:54.592884 0'0 2016-08-01 15:29:08.245671 1.24 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:27.672125 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:20:27.672045 0'0 2016-08-01 15:19:38.509289 2.27 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:47.534822 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:29:47.534754 0'0 2016-08-01 15:29:08.245685 1.23 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:19.671700 0'0 54:16 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:20:19.671609 0'0 2016-08-01 15:19:38.509275 2.20 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:47.591203 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:30:47.591108 0'0 2016-08-01 15:29:08.245579 1.22 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:48.019461 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:20:48.019393 0'0 2016-08-01 15:19:38.509261 2.21 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:41.533804 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:29:41.533722 0'0 2016-08-01 15:29:08.245594 1.21 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:16.670712 0'0 54:16 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:20:16.670643 0'0 2016-08-01 15:19:38.509247 2.22 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:42.534279 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:29:42.534163 0'0 2016-08-01 15:29:08.245608 1.20 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:46.019284 0'0 54:15 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:20:46.019139 0'0 2016-08-01 15:19:38.509233 2.23 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:48.591254 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:30:48.591130 0'0 2016-08-01 15:29:08.245623 1.1f 1 0 0 0 0 1073741824 256 256 active+clean 2016-08-02 15:20:15.671370 47'256 54:263 [2,3] 2 [2,3] 2 47'256 2016-08-02 15:20:15.671302 0'0 2016-08-01 15:19:38.509219 2.1c 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:43.590243 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:30:43.589768 0'0 2016-08-01 15:29:08.245500 1.1e 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:45.021926 0'0 54:15 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:20:45.021827 0'0 2016-08-01 15:19:38.509204 2.1d 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:34.532473 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:29:34.532367 0'0 2016-08-01 15:29:08.245515 1.1d 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:03.666886 0'0 54:16 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:20:03.666821 0'0 2016-08-01 15:19:38.509190 2.1e 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:37.532996 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:29:37.532911 0'0 2016-08-01 15:29:08.245541 1.1c 1 0 0 0 0 1073741824 256 256 active+clean 2016-08-02 15:20:02.667557 47'256 54:264 [2,3] 2 [2,3] 2 47'256 2016-08-02 15:20:02.667448 0'0 2016-08-01 15:19:38.509175 2.1f 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:46.590609 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:30:46.590526 0'0 2016-08-02 15:30:46.590526 1.1b 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:44.020668 0'0 54:15 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:20:44.020608 0'0 2016-08-01 15:19:38.509161 2.18 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:19.586045 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:30:19.585976 0'0 2016-08-01 15:29:08.245440 1.1a 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:43.017294 0'0 54:15 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:20:43.017226 0'0 2016-08-01 15:19:38.509146 2.19 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:31.532267 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:29:31.532139 0'0 2016-08-01 15:29:08.245456 1.19 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:40.017378 0'0 54:15 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:20:40.017326 0'0 2016-08-01 15:19:38.509129 2.1a 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:39.591313 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:30:39.591245 0'0 2016-08-01 15:29:08.245470 1.18 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:37.016709 0'0 54:15 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:20:37.016652 0'0 2016-08-01 15:19:38.509100 2.1b 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:32.531496 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:29:32.531434 0'0 2016-08-02 15:29:32.531434 1.17 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:19:57.665285 0'0 54:16 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:19:57.665224 0'0 2016-08-01 15:19:38.509084 2.14 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:10.586841 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:30:10.586756 0'0 2016-08-01 15:29:08.245376 1.16 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:32.016391 0'0 54:15 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:20:32.016241 0'0 2016-08-01 15:19:38.509070 2.15 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:27.530097 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:29:27.530028 0'0 2016-08-01 15:29:08.245392 1.15 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:31.015355 0'0 54:15 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:20:31.015219 0'0 2016-08-01 15:19:38.509040 2.16 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:11.584182 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:30:11.584093 0'0 2016-08-02 15:30:11.584093 1.14 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:30.014851 0'0 54:15 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:20:30.014794 0'0 2016-08-01 15:19:38.509024 2.17 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:30:18.584840 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:30:18.584785 0'0 2016-08-01 15:29:08.245421 1.13 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:28.015086 0'0 54:15 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:20:28.015011 0'0 2016-08-01 15:19:38.509009 2.10 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:20.529098 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:29:20.529021 0'0 2016-08-01 15:29:08.245302 1.12 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:27.014217 0'0 54:15 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:20:27.014153 0'0 2016-08-01 15:19:38.508994 2.11 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:23.530698 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:29:23.530327 0'0 2016-08-01 15:29:08.245321 1.11 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:24.013906 0'0 54:15 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:20:24.013824 0'0 2016-08-01 15:19:38.508975 2.12 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:51.581043 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:29:51.580984 0'0 2016-08-01 15:29:08.245339 1.10 1 0 0 0 0 1073741824 256 256 active+clean 2016-08-02 15:20:14.548071 47'256 54:247 [3,2] 3 [3,2] 3 47'256 2016-08-02 15:20:14.547890 47'256 2016-08-02 15:20:14.547890 2.13 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:54.581610 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:29:54.581547 0'0 2016-08-01 15:29:08.245353 1.f 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:20:04.009828 0'0 54:15 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:20:04.009750 0'0 2016-08-02 15:20:04.009750 2.c 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:48.580315 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:29:48.580263 0'0 2016-08-02 15:29:48.580263 1.e 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:19:58.009295 0'0 54:15 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:19:58.009212 0'0 2016-08-02 15:19:58.009212 2.d 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:50.580817 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:29:50.580723 0'0 2016-08-02 15:29:50.580723 1.d 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:19:57.007666 0'0 54:15 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:19:57.007585 0'0 2016-08-01 15:19:38.508907 2.e 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:16.528032 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:29:16.527970 0'0 2016-08-02 15:29:16.527970 1.c 1 0 0 0 0 1073741824 256 256 active+clean 2016-08-02 15:19:56.665801 47'256 54:235 [2,3] 2 [2,3] 2 47'256 2016-08-02 15:19:56.665753 0'0 2016-08-01 15:19:38.508887 2.f 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:17.527996 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:29:17.527910 0'0 2016-08-01 15:29:08.245280 1.b 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:19:53.664625 0'0 54:16 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:19:53.664577 0'0 2016-08-01 15:19:38.508853 2.8 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:08.602350 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:29:08.602109 0'0 2016-08-01 15:29:08.245167 1.a 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:19:53.007032 0'0 54:15 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:19:53.006988 0'0 2016-08-01 15:19:38.508823 2.9 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:09.526604 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:29:09.526536 0'0 2016-08-01 15:29:08.245183 1.9 1 0 0 0 0 1073741824 256 256 active+clean 2016-08-02 15:19:52.663973 47'256 54:242 [2,3] 2 [2,3] 2 47'256 2016-08-02 15:19:52.663857 0'0 2016-08-01 15:19:38.508805 2.a 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:43.579716 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:29:43.579651 0'0 2016-08-01 15:29:08.245202 1.8 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:19:51.006903 0'0 54:15 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:19:51.006798 0'0 2016-08-02 15:19:51.006798 2.b 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:10.527319 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:29:10.527199 0'0 2016-08-01 15:29:08.245217 1.7 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:19:49.006770 0'0 54:15 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:19:49.006660 0'0 2016-08-01 15:19:38.508771 2.4 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:33.577360 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:29:33.577290 0'0 2016-08-01 15:29:08.245095 1.6 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:19:50.663220 0'0 54:16 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:19:50.663152 0'0 2016-08-01 15:19:38.508751 2.5 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:39.577866 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:29:39.577774 0'0 2016-08-01 15:29:08.245115 1.5 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:19:46.005775 0'0 54:15 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:19:46.005688 0'0 2016-08-01 15:19:38.508734 2.6 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:40.578494 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:29:40.578399 0'0 2016-08-01 15:29:08.245130 1.4 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:19:48.662852 0'0 54:16 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:19:48.662787 0'0 2016-08-01 15:19:38.508713 2.7 1 0 0 0 0 1073741824 256 256 active+clean 2016-08-02 15:29:42.578799 47'256 54:229 [0,1] 0 [0,1] 0 47'256 2016-08-02 15:29:42.578758 0'0 2016-08-01 15:29:08.245150 1.3 1 0 0 0 0 1073741824 256 256 active+clean 2016-08-02 15:19:46.670449 53'256 54:241 [2,3] 2 [2,3] 2 53'256 2016-08-02 15:19:46.670378 0'0 2016-08-01 15:19:38.508695 2.0 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:21.575113 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:29:21.575030 0'0 2016-08-01 15:29:08.244991 1.2 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:19:43.010685 0'0 54:15 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:19:43.010519 0'0 2016-08-02 15:19:43.010519 2.1 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:23.575656 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:29:23.575569 0'0 2016-08-01 15:29:08.245025 1.1 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:19:41.005466 0'0 54:15 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:19:41.005319 0'0 2016-08-01 15:19:38.508656 2.2 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:29:28.576463 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:29:28.576382 0'0 2016-08-01 15:29:08.245040 1.0 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:19:40.114171 0'0 54:15 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:19:40.114013 0'0 2016-08-01 15:19:38.508575 2.3 1 0 0 0 0 1073741824 256 256 active+clean 2016-08-02 15:29:31.577299 47'256 54:232 [0,1] 0 [0,1] 0 47'256 2016-08-02 15:29:31.577183 0'0 2016-08-01 15:29:08.245071 2.63 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:10.607286 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:32:10.607239 0'0 2016-08-01 15:29:08.246700 1.60 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:17.703334 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:22:17.703265 0'0 2016-08-01 15:19:38.510197 2.62 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:07.607203 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:32:07.607109 0'0 2016-08-01 15:29:08.246682 1.61 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:20.701252 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:22:20.701192 0'0 2016-08-02 15:22:20.701192 2.61 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:43.561223 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:31:43.561154 0'0 2016-08-01 15:29:08.246665 1.62 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:24.702002 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:22:24.701891 0'0 2016-08-01 15:19:38.510226 2.60 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:04.606321 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:32:04.606235 0'0 2016-08-01 15:29:08.246647 1.63 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:29.703012 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:22:29.702939 0'0 2016-08-01 15:19:38.510240 2.67 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:16.608799 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:32:16.608701 0'0 2016-08-02 15:32:16.608701 1.64 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:35.703867 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:22:35.703748 0'0 2016-08-01 15:19:38.510254 2.66 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:01.564532 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:32:01.564443 0'0 2016-08-01 15:29:08.246750 1.65 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:51.706619 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:22:51.706563 0'0 2016-08-01 15:19:38.510269 2.65 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:31:54.563278 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:31:54.563202 0'0 2016-08-01 15:29:08.246734 1.66 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:53.707851 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:22:53.707697 0'0 2016-08-01 15:19:38.510283 2.64 1 0 0 0 0 1073741824 256 256 active+clean 2016-08-02 15:32:13.608675 47'256 54:232 [0,1] 0 [0,1] 0 47'256 2016-08-02 15:32:13.608572 0'0 2016-08-01 15:29:08.246716 1.67 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:22.039056 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:22:22.038978 0'0 2016-08-01 15:19:38.510297 2.6b 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:10.566021 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:32:10.565941 0'0 2016-08-01 15:29:08.246846 1.68 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:55.707704 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:22:55.707646 0'0 2016-08-01 15:19:38.510311 2.6a 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:20.609588 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:32:20.609529 0'0 2016-08-02 15:32:20.609529 1.69 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:59.708940 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:22:59.708824 0'0 2016-08-01 15:19:38.510325 2.69 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:19.609483 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:32:19.609425 0'0 2016-08-02 15:32:19.609425 1.6a 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:23:01.709217 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:23:01.709162 0'0 2016-08-01 15:19:38.510339 2.68 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:04.564806 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:32:04.564738 0'0 2016-08-01 15:29:08.246800 1.6b 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:23:04.709752 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:23:04.709679 0'0 2016-08-01 15:19:38.510353 2.6f 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:24.610627 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:32:24.610564 0'0 2016-08-02 15:32:24.610564 1.6c 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:23:05.709534 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:23:05.709472 0'0 2016-08-02 15:23:05.709472 2.6e 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:23.610354 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:32:23.610305 0'0 2016-08-01 15:29:08.246904 1.6d 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:23:07.710000 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:23:07.709904 0'0 2016-08-01 15:19:38.510382 2.6d 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:22.610292 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:32:22.610248 0'0 2016-08-01 15:29:08.246878 1.6e 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:23:10.710685 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:23:10.710516 0'0 2016-08-01 15:19:38.510396 2.6c 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:11.566366 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:32:11.566323 0'0 2016-08-01 15:29:08.246862 1.6f 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:23:16.712274 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:23:16.712197 0'0 2016-08-01 15:19:38.510410 2.73 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:41.571176 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:32:41.571128 0'0 2016-08-02 15:32:41.571128 1.70 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:23:19.713001 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:23:19.712909 0'0 2016-08-01 15:19:38.510424 2.72 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:27.569864 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:32:27.569710 0'0 2016-08-01 15:29:08.246991 1.71 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:23:26.713491 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:23:26.713422 0'0 2016-08-01 15:19:38.510438 2.71 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:18.568672 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:32:18.568588 0'0 2016-08-02 15:32:18.568588 1.72 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:24.040349 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:22:24.040277 0'0 2016-08-01 15:19:38.510453 2.70 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:12.566371 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:32:12.566280 0'0 2016-08-01 15:29:08.246956 1.73 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:23:29.714561 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:23:29.714445 0'0 2016-08-02 15:23:29.714445 2.77 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:27.610635 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:32:27.610593 0'0 2016-08-02 15:32:27.610593 1.74 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:23:32.714755 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:23:32.714684 0'0 2016-08-02 15:23:32.714684 2.76 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:26.611291 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:32:26.611227 0'0 2016-08-01 15:29:08.247091 1.75 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:23:33.715428 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:23:33.715349 0'0 2016-08-02 15:23:33.715349 2.75 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:45.572048 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:32:45.571975 0'0 2016-08-01 15:29:08.247073 1.76 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:31.041825 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:22:31.041704 0'0 2016-08-01 15:19:38.510520 2.74 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:44.572207 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:32:44.572150 0'0 2016-08-01 15:29:08.247037 1.77 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:35.042080 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:22:35.042000 0'0 2016-08-02 15:22:35.042000 2.7b 1 0 0 0 0 1073741824 256 256 active+clean 2016-08-02 15:32:34.617002 47'256 54:257 [0,1] 0 [0,1] 0 47'256 2016-08-02 15:32:34.616942 0'0 2016-08-01 15:29:08.247175 1.78 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:23:35.716106 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:23:35.715972 0'0 2016-08-01 15:19:38.510549 2.7a 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:31.612522 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:32:31.612451 0'0 2016-08-01 15:29:08.247158 1.79 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:23:36.716014 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:23:36.715941 0'0 2016-08-01 15:19:38.510563 2.79 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:29.612099 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:32:29.612043 0'0 2016-08-01 15:29:08.247141 1.7a 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:36.042464 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:22:36.042401 0'0 2016-08-01 15:19:38.510577 2.78 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:28.611958 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:32:28.611909 0'0 2016-08-01 15:29:08.247124 1.7b 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:37.042793 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:22:37.042709 0'0 2016-08-01 15:19:38.510591 2.7f 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:52.574411 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:32:52.574340 0'0 2016-08-01 15:29:08.247244 1.7c 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:38.042889 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:22:38.042820 0'0 2016-08-02 15:22:38.042820 2.7e 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:48.572784 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:32:48.572735 0'0 2016-08-02 15:32:48.572735 1.7d 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:40.044293 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:22:40.044222 0'0 2016-08-01 15:19:38.510621 2.7d 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:36.613750 0'0 54:10 [0,1] 0 [0,1] 0 0'0 2016-08-02 15:32:36.613685 0'0 2016-08-01 15:29:08.247208 1.7e 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:23:38.717317 0'0 54:15 [2,3] 2 [2,3] 2 0'0 2016-08-02 15:23:38.717254 0'0 2016-08-01 15:19:38.510635 2.7c 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:32:47.572402 0'0 54:10 [1,0] 1 [1,0] 1 0'0 2016-08-02 15:32:47.572356 0'0 2016-08-01 15:29:08.247192 1.7f 0 0 0 0 0 0 0 0 active+clean 2016-08-02 15:22:44.044340 0'0 54:14 [3,2] 3 [3,2] 3 0'0 2016-08-02 15:22:44.044243 0'0 2016-08-01 15:19:38.510650 pool 2 10 0 0 0 0 10166992896 2424 2424 pool 1 9 0 0 0 0 9663676416 2304 2304 sum 19 0 0 0 0 19830669312 4728 4728 osdstat kbused kbavail kb hb in hb out 3 9476312 998164 10474476 [0,1,2] [] 2 9476412 998064 10474476 [0,1,3] [] 1 9967672 506804 10474476 [0,2,3] [] 0 9967672 506804 10474476 [1,2,3] [] sum 38888068 3009836 41897904
> It seems that %USED and MAX AVAIL are calculated by looking at the actual disk usage reported by each osd in the tree used by that pool. USED reflects the nominal amount stored by the pool without accounting for overhead or replication (1000 MB of objects is 1000 MB of USED even if the actual on disk usage is much higher due to replication and overhead). > > In short, you cannot calculate %USED from MAX_AVAIL and USED. currently, %USED is calculated using "the raw space used by the pool" / "total raw space offered by all OSDs", as comment 3 pointed out. but the problem of this algorithm is: it takes the OSDs not assigned to the pool into consideration. which renders the percentage less useful in the context of a metrics of a certain pool. if we want to use %USED as an indicator so that we can warn the user that "you need to add more OSDs into the pool" if %USED > a-predefined-threshold, then this would be a bad choice. and MAX_AVAIL is calculated using "the total available size calculated according to the rule of this pool". so, to make %USED more useful, maybe we should calculate it by utilization = (USED * 100) / (USED + MAX AVAIL) as comment 1 suggests?
It looks like 'ceph pg dump' and 'ceph df' work differently. I think Kefu is talking about the code for 'ceph pg dump' (which only uses the pool stats rather than the actual filesystem usage).
Ok, ignore what I said above in comment 20 (sorry, I got confused!).
upstream tracker added: http://tracker.ceph.com/issues/16933 PR posted at https://github.com/ceph/ceph/pull/10584
Verified Percentage Calculation of Used space is working as per the request raised by Darshan Steps : http://pastebin.test.redhat.com/408499
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2016-1898.html