Created attachment 1863288 [details] Persistent Write Log Cache User Guide Description of feature: 1.The Persistent Write Log Cache (PWL) provides a persistent, fault-tolerant write-back cache for librbd-based RBD clients. 2.This cache uses a log-ordered write-back design which maintains checkpoints internally so that writes that get flushed back to the cluster are always crash consistent. Even if the client cache is lost entirely, the disk image is still consistent, but the data will appear to be stale. 3.This cache can be used with PMEM or SSD as a cache device. For PMEM, the cache mode is called replica write log (rwl). For SSD, the cache mode is called ssd. Feature benefits: 1.PWL can provide very high performance when the cache is not full. So, the larger the cache, the longer the duration of high performance. 2.PWL cache provide persistence feature. And it is not much slower than RBD cache. RBD cache is faster but volatile. It can’t guarantee data order and persistence. 3.In a steady state where the cache is full, the performance is affected by the number of IOS in flight. For example, PWL can provide higher performance at low io_depth, but at high io_depth, such as when the number of IO is greater than 32, the performance is often worse than that in scenes without cache. Configuration Pre-requisites： 1.Opreation System: Linux; 2.Ceph version >= pacific; 3.Client side disk: pmem or SSD; 4.A healthy Ceph cluster; 5.An image with exclusive-lock feature(default); 6.It’s better to disable rbd_cache; Enabling cache: edit ceph configure, for example: vim /etc/ceph/ceph.conf: [client] rbd_cache = false rbd_persistent_cache_mode = rwl (or ssd) rbd_plugins = pwl_cache rbd_persistent_cache_size = 1073741824 rbd_persistent_cache_path = /mnt/pmem/cache/ Disabling cache(default is disable): Comment out the configuration. It is disable by default. Also, you can remove it or set rbd_persistent_cache_mode disable. vim /etc/ceph/ceph.conf: [client] rbd_cache = false #rbd_persistent_cache_mode = disable #rbd_plugins = pwl_cache rbd_persistent_cache_size = 1073741824 rbd_persistent_cache_path = /mnt/pmem/cache/ Best case scenarios for PMem caching: 1.The nonvolatile and good performance of RWL can replace rbd_cache as the client side cache of virtual machine. This requires code support, because when the virtual machine in ceph enable cache, rbd will enable rbd_cache by default. so, not support now. 2.RWL cache provide low latency. It also has a stable low latency for burst I/Os, so it is suitable for those scenarios with high requirements for stable low latency. 3.RWL also has high continuous and stable performance improvement in scenarios with low IO depth or not too much inflight io. Best case scenarios for SSD caching: The advantage of SSD mode are similar to RWL. SSD hardware is relatively cheap and popular, but its performance is a little lower than PMEM. Additional info: Please see the attachment for more details. Include: 1.More detailed steps. 2.How to check in each step. 3.How to install pmem. 4.How to use PWL command.
Please specify the severity of this bug. Severity is defined here: https://bugzilla.redhat.com/page.cgi?id=fields.html#bug_severity.
@congmin, Is the scope of the BZ is for documentation improvements for 5.2 or as mentioned below for replica function which is under development? currently this feature is Tech preview for 5.1 3.This cache can be used with PMEM or SSD as a cache device. For PMEM, the cache mode is called replica write log (rwl). At present, only local cache is supported, and the replica function is under development. For SSD, the cache mode is called ssd.
(In reply to Preethi from comment #2) > @congmin, Is the scope of the BZ is for documentation improvements for 5.2 > or as mentioned below for replica function which is under development? > currently this feature is Tech preview for 5.1 > > 3.This cache can be used with PMEM or SSD as a cache device. For PMEM, the > cache mode is called replica write log (rwl). At present, only local cache > is supported, and the replica function is under development. For SSD, the > cache mode is called ssd. It is doc for 5.1. But I mentioned the unfinished replica function progress. Should I removed this statement?
@Preethi Sorry, I didn't notice the advance field. It is hidden by default and has a default value. Now I have changed the target version and DOC tpye.
(In reply to Congmin from comment #3) > (In reply to Preethi from comment #2) > > @congmin, Is the scope of the BZ is for documentation improvements for 5.2 > > or as mentioned below for replica function which is under development? > > currently this feature is Tech preview for 5.1 > > > > 3.This cache can be used with PMEM or SSD as a cache device. For PMEM, the > > cache mode is called replica write log (rwl). At present, only local cache > > is supported, and the replica function is under development. For SSD, the > > cache mode is called ssd. > > It is doc for 5.1. But I mentioned the unfinished replica function > progress. Should I removed this statement? yes, lets remove the replica function content to avoid confusion for Doc team. Since, Its doc enhancement. I will change the component to Documentation to take it for closure asap.
(In reply to Congmin from comment #3) > (In reply to Preethi from comment #2) > > @congmin, Is the scope of the BZ is for documentation improvements for 5.2 > > or as mentioned below for replica function which is under development? > > currently this feature is Tech preview for 5.1 > > > > 3.This cache can be used with PMEM or SSD as a cache device. For PMEM, the > > cache mode is called replica write log (rwl). At present, only local cache > > is supported, and the replica function is under development. For SSD, the > > cache mode is called ssd. > > It is doc for 5.1. But I mentioned the unfinished replica function > progress. Should I removed this statement? Yes, better to avoid confusion.
Created attachment 1863776 [details] Persistent Write Log Cache User Guide
> yes, lets remove the replica function content to avoid confusion for Doc > team. Since, Its doc enhancement. I will change the component to > Documentation to take it for closure asap. @Preethi I remove the replica content. I also updated the attachment doc, then uploaded. Do I need to remove the previous attachment? I don't know how to remove it.
(In reply to Congmin from comment #8) > > yes, lets remove the replica function content to avoid confusion for Doc > > team. Since, Its doc enhancement. I will change the component to > > Documentation to take it for closure asap. > > > @Preethi I remove the replica content. I also updated the attachment doc, > then uploaded. Do I need to remove the previous attachment? I don't know how > to remove it. Np. Will use the latest attached. Thanks Congmin.
We can consider this BZ as continuation of this BZ https://bugzilla.redhat.com/show_bug.cgi?id=1980747 for doc enhancement. Additional content can be referred from here.
(In reply to Preethi from comment #10) > We can consider this BZ as continuation of this BZ > https://bugzilla.redhat.com/show_bug.cgi?id=1980747 for doc enhancement. > Additional content can be referred from here. Yes, there is a connection between the two. If you think it's feasible, can you help to do it?
Sure. I will work with doc team for implementation and take it for closure. Thanks Congmin.
QE do not have updates that its GA release and not a Tech preview for 5.1 as "tech preview" word has been removed from the doc. Rest looks fine. We will wait for the confirmation.
(In reply to Preethi from comment #19) > QE do not have updates that its GA release and not a Tech preview for 5.1 as > "tech preview" word has been removed from the doc. Rest looks fine. We will > wait for the confirmation. @Preethi Do we need to change the "doc type"? Now it is "tech preview", but I don't know what to change.
Content looks good. Am waiting for final conclusion whether its TP or GA support for 5.1 release to move it to be verified state. Awaiting for confirmation.
We will go with TP for now as discussion are still in progress and 5.1 GA is near by. Hence, moving this to verified state.