Hide Forgot
Description of problem: If OSD's are marked out, then its not retaining its old ReWeight Value when again its marked in. It is setting to default weight. Version-Release number of selected component (if applicable): ceph version 0.94.5-6redhat1trusty How reproducible: Always Steps to Reproduce: Use reweight-by-utilization to change the OSD's reweight values $ sudo ceph osd df ID WEIGHT REWEIGHT SIZE USE AVAIL %USE VAR 0 1.00000 0 0 0 0 0 0 1 0.89999 0 0 0 0 0 0 2 0.89999 0 0 0 0 0 0 15 0.09000 0.52400 29985M 21113M 8871M 70.41 0.95 16 0.09000 0.37169 29985M 22099M 7885M 73.70 0.99 17 0.03000 1.00000 29985M 18194M 11790M 60.68 0.82 18 0.03000 0.78389 29985M 24511M 5474M 81.74 1.10 19 0.03000 1.00000 29985M 24040M 5944M 80.17 1.08 20 0.03000 1.00000 29985M 20944M 9040M 69.85 0.94 21 0.03000 0.95001 29985M 23394M 6591M 78.02 1.05 22 0.03000 1.00000 29985M 23536M 6448M 78.49 1.06 23 0.03000 0 0 0 0 0 0 3 0.89999 0 0 0 0 0 0 4 0.89999 0 0 0 0 0 0 5 0.89999 0 0 0 0 0 0 6 0.89999 0 0 0 0 0 0 7 0.89999 0 0 0 0 0 0 8 0.89999 0 0 0 0 0 0 9 0.89999 0 0 0 0 0 0 10 0.89999 0 0 0 0 0 0 11 0.89999 0 0 0 0 0 0 12 0.89999 0 0 0 0 0 0 13 0.89999 0 0 0 0 0 0 14 0.89999 0 0 0 0 0 0 TOTAL 234G 173G 62046M 74.13 MIN/MAX VAR: 0/1.10 STDDEV: 6.97 2. Marked the OSD out. $ sudo ceph osd out osd.18 marked out osd.18. 3. Marked the OSD in. $ sudo ceph osd in osd.18 marked in osd.18. [ubuntu@magna009 ~]$ sudo ceph osd df ID WEIGHT REWEIGHT SIZE USE AVAIL %USE VAR 0 1.00000 0 0 0 0 0 0 1 0.89999 0 0 0 0 0 0 2 0.89999 0 0 0 0 0 0 15 0.09000 0.52400 29985M 21141M 8843M 70.51 1.01 16 0.09000 0.37169 29985M 22662M 7323M 75.58 1.08 17 0.03000 1.00000 29985M 18568M 11417M 61.92 0.89 18 0.03000 1.00000 29985M 21146M 8838M 70.52 1.01 19 0.03000 1.00000 29985M 20723M 9261M 69.11 0.99 20 0.03000 1.00000 29985M 20251M 9733M 67.54 0.97 21 0.03000 0.95001 29985M 21783M 8201M 72.65 1.04 22 0.03000 1.00000 29985M 21236M 8749M 70.82 1.01 23 0.03000 0 0 0 0 0 0 3 0.89999 0 0 0 0 0 0 4 0.89999 0 0 0 0 0 0 5 0.89999 0 0 0 0 0 0 6 0.89999 0 0 0 0 0 0 7 0.89999 0 0 0 0 0 0 8 0.89999 0 0 0 0 0 0 9 0.89999 0 0 0 0 0 0 10 0.89999 0 0 0 0 0 0 11 0.89999 0 0 0 0 0 0 12 0.89999 0 0 0 0 0 0 13 0.89999 0 0 0 0 0 0 14 0.89999 0 0 0 0 0 0 TOTAL 234G 163G 72368M 69.83 MIN/MAX VAR: 0/1.08 STDDEV: 3.62 Actual results: osd.18 is not retaining its old REWEIGHT value which was 0.78389, when i am again marking the OSD in. Expected results: osd.18 should have retained the same value after marked in 0.78389 Additional info:
This is actually intentional, we do remember the weight if the osd is marked out automatically, just not if marked out manually. I'll change this to an rfe for the manual case.
This is intentional behavior and there are many better options for handling OSD balance problems these days that should be used instead of the CRUSH reweights anyway (thank you, balancer module and upmap!).